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ABSTRACT 
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CONCLUSIONS AND RECOMMENDATIONS 



The results o£ the field testing o£ the KIT would be of no value 
unless the KIT developers learned things that would enable them to im- 
prove the product or its dissemination. Criticisms of the KIT and rec- 
ommendations for its improvement were solicited ^t many stages of all 

N 

the field-test strategies. What was learned (or feleamed) and the ac- 
tions taken in response to what was learned are itemized below: 

1. The principal is a pivotal decision maker regarding instructional 
programs in the school. The KIT will continue to be addressed primarily 
to principals. 

2. Principals presently feel pressures for curricular reform from 
district administrators, teachers, and parents. Our approach to points- 
of-view needs assessment will be maintained as a realistic approach to 
addressing the felt pressures for change. 

3. The principal does not consult very much with his teaching 
staff or the parents in matters of planning curriculum because he does 
not know how to do it effectively. The card- sort, points -of -view pro- 
cedure will be expanded upon (see below) to make this necessary in- 
volvement of Various constituencies even easier and more attractive to 
the principal. 

4. Most schools have some sort of innovative project of an instruc- 
tional nature under way. Since innovative programs particularly need 
evaluation, the primary aims of the KIT will meet a real need of the 
principals . 

5. Principals are "cautious" in their attitudes about standardized 
tests. For this reason, they utilize tests less than optimally. The 
KIT will continue and expand its efforts to make test results more use- 
ful for everyday decision making by the principal. 

6. Most tests en^loyed in schools are mandated by the district or 
by the state. There is still room for decisions by principals, however, 
and this room may be in the evaluation of special programs of the 
schools. For this reason, alone, the inclusion of the MEAN test evalu- 
ations is justified. 



7. Scores o£ standardized tests are not frequently used for pro- 
gram or school evaluation. The principal's needs for instruction and 
the need for the development of procedures for program assessment using 
published tests are evident. The KIT will continue to stress the addi- 
tional values inherent in such evaluation devices and will develop im- 
proved methods for utilizing the tests for program and school evaluation. 

8. Principals feel that teacher and guidance personnel attitudes 
towards standardized tests are fairly positive. This perceived positive 
attitude bodes well for the KIT's continued stress on their utilization 
for program evaluation. The comparatively lower attitudes of the stu- 
dents reflect most o£ the tests ' relatively poor showing on the exam- 
inee appropriateness dimension of the test evaluations. 

9. The principals do not have much confidence in using national 
norms for published tests. They want school norms and expressed the de- 
sire for differentiated school norms. Provision for some differentiated 
school norms and for procedures for estimating such norms will be made 
in the published version of the KIT. The Center is continuing to exert 
pressure on test publishers to further investigate the utilization of 
differentiated school norms. 

10. The goals (as printed on tlie sorting cards) seem to most prin- 
cipals and teachers to be fairly exhaustive and reasonably organized. 

With the expansion of some particular behaviors in the goal descriptions, 

tlie goal cards will remain essentially as they were field tested. 

11. The vocabulary on the goal cards was difficult, especially for 
most parents and community members. The goal cards will undergo one 
more field test, this time with lower -middle class parents, with the in- 
tent of soliciting their advice on translating the vocabulary to a sim- 
pler level, while maintaining the intended meanings. It is expected 
that this will result in cards and goals that can be understood by a 
larger spectrum of the population. 

12. While half of the field-test participants desired the goals to 
be stated at a more detailed level, the other half wished them at a more 
global level. A compromise will be struck by leaving the goals at the 
level of their field testing. 



13. The goal -sorting procedure was found to be easy and not odious 
to the participants . The card-sort procedure will be maintained, with 
some changes noted above and below. 

14. Ten decks of cards did not seem sufficient for the principal 
to efficiently sample from his selected constituencies. The published 
KIT will have more decks included in the package (number undetermined) 
and the publisher will be urged to make available, under separate order, 
additional decks of goal -sort cards. 

15. Many participants experienced difficulty in the card-sorting 
procedure when asked to consider "children in general." Instructions 
in Booklet II will be revised so that the card-sort can be done for a 
child at any particular grade level. This will eliminate mucli of the 
confusion expressed by teachers and parents. 

16. The introduction to the card-sort procedure should not have 
been left solely in the hands of the principal. New illustrated and 
more detailed printed instructions to the card-sort procedure will be 
included in the published version of the KIT, which will minimize the 
confusion caused by lack of clarity on the part of the principal as he 
introduces the card- sort procedure. 

17. The tallying process, based upon the card-sort, was found to 
be tedious and difficult. Two alternatives are being prepared for inclu- 
sion in the KIT which will minimize the difficulty and tediousness of the 
tallying procedure. First, the tally sheets will not be number-coded by 
goals, but will list the goals in alphabetical order, so that the card- 
tally sheet transfer is a one-step operation rather than a two-step 
operation. Second, the procedure utilized by the innovative principal 
in School B of the case-study sample, in which the teachers did the 
tallying in a group, exchanging ideas and opinions (which tlie project 
team found to be most stimulating for all concerned) will be provided as 
an option for sampling of teachers. This necessitates that the KIT's 
approach to the card-sort procedure will be trichotomized, approaches 
for the principal, for teachers, and for parents (see next item). 

18. The return and completion of parents' card decks was dis- 
appointingly low. Principals found it difficult to sample well from the 
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parent groups and then to get the parents who did cooperate to complete 
the card-sort. To alleviate this problem, the goal-rating procedure will 
be expanded to a simple rating form questionnaire, so that the principal 
can mail out self-instructional questionnaires to all the parents, with 
the expectancy that his returns will be much greater. The questionnaires 
will be highly graphic and interesting to the parent and will be avail- 
able in quantity from the publisher, under separate order, according to 
the plan. 

19. Most principals could correctly employ the MEAN ratings to 
select the best tests for their use. The test evaluation format will 
remain as it was field tested. 

20. Most of the principals exliibited complete surface understand- 
ing of the four I^IEAN criteria. The test evaluations will not be altered 
in order to simplify things for the principal; if most can understand 

it at the level provided, we will maintain the same level of detail as 
in the field test. 

21. The test evaluation section was found to be clear and usable. 
The expectations of confusion on the part of the typical principal did 
not materialize; the appendix will remain as in the field-test version. 

22. The test-administration-ordering section was considered well- 
packaged by the principals and thought of as providing information in 

a useful form, even tliough the information was not novel to them. The 
sections referred to will remain as in the field-test version. 

23. Most principals do not favor sampled- system assessment, but 
prefer the less efficient (but more information -providing) student- 
system assessment method. Tlie perceived (but not completely verid- 
ical) favoring of a sampled- system assessment by the Center will be 
eliminated. Since both methods have distinct advantages, both will 
be given equal attention, with the advantages of both discussed. 

24. Tlie discussion of decision models, and decision rules was 
judged to be very clear by the principals. Counter to the Program's 
expectations, these sections were not too difficult for the users, so 
they will not be written down to a simpler (and less informative) 
level. 

•»fc 

t - - - - - ^ ^ 
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,f 25. Most principals want greater explanation of the numbers in 

i the utility and. value tables and want greater explanation of the deriva- 

[ tions of the procedures. A slight expansion of these sections will be 

\ accomplished and included in the published version of the KIT. It is 

apparent .that we aimed a bit too low in this section, being too cau- 
tious about overvdielming the principal, when, in fact, the principal 
needs some documentation so that he can have a ground for the confi- 
dence he needs in order to utilize the product. 

26. While the leadership qualities of the principal appear to 
be important, the informal social structure among the teachers does not 
appear to be important to the successful implementation of the KIT. 

The KIT will continue to be addressed to the principal, and no provision 

will be made at .this time for organizational problems within the school. 
In an effort to minimize the degree to which the KIT rests. on the unpre- 
dictable leadership .talents of elementary school principals , the KIT's 
directions will be made much more extensive, directive, and specific. 
While this may limit the creative ways in which the KIT might be used, 
more explicit directions should reduce the variability of the KIT's in- 
troduction. These revisions will give detailed instructions on how to 
introduce the KIT, a timetable for use of the KIT, and a more compre- 
hensive introduction to the design of the evaluation KIT. 

DESCRIPTION OF THE KIT 

School administrators are faced, more and more with situations 
vhich require that accountable decisions be made on the basis of very 
limited information. Since these decisions relate directly to the 
effectiveness of the school, it is vital that procedures be developed 
for selecting, collecting, analyzing, and providing information that 
will.be of use to .principals in making educational decisions. jThe 
information. contained in published research studies, however, cer- 
tainly does not . include the entire range of school dimensions that 
school principals must consider in making their decisions; and the 
form. of such reports is not readily amenable to utilization by de- 
cision makers. 
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The Sc±iool Evaluation Program was designed to meet this need o£ 
decision makers. The Program has designed a portion o£ a do-it-your- 
sel£ in£ormation system £or elementary school administrators to obtain 
in£ormation about student per£ormance and the variables by which it is 
in£luenced. The system provides this in£ormation to the principal in 
a manner that will enable him to use it e££ectively in making decisions 
The principal is seen as the prime user o£ the system since he has to 
make most o£ the major daily decisions concerning school policy and 
procedures. The present trend toward decentralization is certainly in- 
creasing his responsibility in this respect. 

Though the in£ormation system is being designed speci£ically £or 
the principal and assumes very little in the way o£ evaluation or mea- 
surement skills, it could also be use£ul to others interested in the 
output o£ schools, such as congressmen, parents, school superinten- 
dents, and teachers. A superintendent, £or instance, could use it to 
examine the relative strengths and weaknesses o£ student performance 
(adjusted £or input) at the di££erent schools in his district. 

In preparation £or constructing this system, the Program has re- 
viewed existing data, evaluation techniques, and assessment measures ; 
and where necessary, has collected additional in£ormation, modi£ied 
methods, and developed innovative procedures. The resulting products 
are intended to provide the school principal with an e££icient sel£- 
help in£ormation system which will help him to assess the needs o£ 
his students, indicate what he might do to £ul£ill those needs and 
to encourage higher levels o£ student per£ormance, and enable him to 
select and then plan and evaluate instructional programs having the 
maximum likelihood o£ success in £illing the needs. Past phases o£ the 
Program have provided preliminary research; and pilot studies have led 
to the development o£ the £irst module o£ the in£ormation system, the 
£ield testing o£ the system prototype, and research directed at system 
re£inement, revision and extension. 

The primary objective o£ the School Evaluation Program is the 
e££icient production o£ a large number o£ trained evaluators capable o£ 
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carrying out evaluations of their local educational programs and fa- 
cilities in a rational and objective manner. It is commonly accepted 
that one way of effective improvement in education is through just such 
valid and meaningful evaluation. This belief is exemplified by the 
evaluation requirements mandated in many of the federally-sponsored 
educational endeavors. These mandates, and others like them at the 
state and municipal levels, provide for external evaluations of the 
educational system. But internal evaluations, providing self-knowledge, 
may be the more dramatic way to effect improvement that will be in- 
ternalized and implemented. 



Careful consideration of the urgent national need for trained 
evaluators and of the necessarily limited numbers of fonnalized pro- 
grams of instruction or training workshops --limitations that can be sur- 
mounted by self- instructional packages- -has led the Program to the 
conclusion that every effort should be made to make available cur- 
rent and valid evaluation procedures and methods to as many of the de- 
cision makers in education as possible. Although the role of the educa- 
tional decision maker is distinct from that of the evaluator, in actual 
practice this separation of roles at the local level is at best improb- 
able. A do-it-yourself, self -instructional guide (KIT) for doing edu- 
cational evaluation was early decided upon as the vehicle by which this 
need could best be met on a massive scale. 



Before general release of the self-instructional materials to prin- 
cipals in the nation, the Program demanded that both intensive and ex- 
tensive study be made of their appropriateness for untrained school ad- 
ministrators , effectiveness as an instructional device, and impact in 
altering and iii5)roving attitudes and practices in school and program 
evaluations. The present report is in response to this demand. 



The vehicle developed to achieve the goal of rapidly producing an 
adequate supply of trained educational evaluators Has taken the fom 
of a series of KIT’s: self- instructional packages which will enable a per- 

son functioning in the evaluator’s role to understand the procedures, 
methods, and underlying bases of educational evaluation, and then to 
cahry out' evaluations in a rig;6rbus ai^ e:^ic^^ 
like the KITs to follow, has been ^di'rected to^^^^^^ 
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school principals. It assumes no. particular laiowledge or experience 
in measurement or evaluation, yet its step-by-step approach will enable 
the novice to make valid evaluations. 

The first product developed and field tested in this self- instruction- 
al approach to the training of education evaluators is the Elementary 
School Evaluation KIT: Needs Assessment . This KIT guides the principal 

step-by-step through a school’s assessment of educational needs, using 
examples, problems, and discussions pertinent to characteristics of the 
elementary school. The five booklets and supporting materials compris- 
ing the KIT can be optimally studied and utilized in about seven months. 

The materials of the KIT attack problems such as (1) how the princi- 
pal can select the information he needs regarding student performance 
that will reflect the views of parents, teachers, and school board mem- 
bers, (2) how the principal can select the tests that will give him the 
kinds of information he needs; tests keyed to the student and school 
needs he has previously determined to be of importance; (3) how the 
principal can effectively collect the needed test information and then 
interpret the test data in light of his school’s particular and unique 
characteristics; and (4) how the principal can decide where to place his 
resources to get the greatest improvement in his school in the future, 
considering the importance of the subject area, the value of given im- 
provements in achievenent , and the probable amount of improvement he can 
expect. 

THE NEED FOR FIELD TESTING OF EDUCATIONAL PRODUCTS 

From the point of view of accountability, any educational product 
released on a large scale and purporting to address a critical issue 
should be adequately, if not exhaustively, pretested to ensure that it 
will result in more good than harm. This is particularly the case when 
the issue being addressed has an (undeservedly) obscure nature about 
it, \diere the reader or user cannot be expect^ to muster knowledgeable 
criticism of the product himself due to his real or imagined limitations . 
When the product addresses the issue of educational evaluation, where 
inadequacies are acutely felt, such is truly the case. 
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During the first two years of its development, the Elementary 
School Evaluation KIT; Needs Assessment has undergone initial field 
testing at the components level with large groups of educators at var- 
ious national meetings and more extensively with 23 principals and 
superintendents throughout the State of California. The California 
educators worked with the Program through the auspices of the Cali- 
fornia Elementary School Administrators Association, under the direc- 
tion of Dr. Edward W. Beaubier. The initial pilot testing resulted 
in an edition of the KIT that was deemed ready for major field testing. 



THE FIELD TEST STRATEGIES 



The field testing of the Elementary School Evaluation KIT ; Needs 
Assessment was conducted as the last stage of the formal try-out of the 
product in order to determine its strengths and to remedy its weakness- 
es before it is made available to the educational public via a commer- 
cial publishing organization. The primary goal of the field test was 
to determine the usefulness and viability of the prototype KIT in an 
environment that was not necessarily conpletely receptive, but was 
characteristic of the environment for which the KIT is intended. An 
additional goal was to measure the extent to which the KIT was imple- 
mented in the school and the cost of this in^lementation; and to esti- 
mate the effectiveness, endurance, and potential of the KIT in the 
school environment. 

The specific objectives of the field-test instruments used to 
collect data from the principals werer (1) to determine whether var- 
ious procedures contained in the KIT had been implemented by the prin- 
cipal; (2) to determine any changes that had occurred in the follow- 
ing areas that could have resulted from use of the KIT- -(a) the atti- 
tude of the principal and his staff toward evaluation, (b) the methods 
used to make decisions relative to the instructional program of the 
school, (c) the understanding of the principal and his staff of the 
evaluation principles on which the KIT is based; and (3) to determine 
the subjective opinion of the principal and his staff toward the con- 
tents of the KIT. 

Originally the field-testing plan envisioned a national represent- 
ative sanple that might to some degree slight representation of Califor- 
nia, the state in which all of the initial pilot testings and trials 
were undertaken. However, due to the opportunity to utilize the Cali- 
fomia schools once again, and in a manner that would increase our 
knowledge about possible variations for the iiiplementation of the KIT 
into schools , a separate sanple of California sch^ was added to the 
planned; national, saiipie.:, ■'r.-? ' ^ ^ 
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T he National Sampling 

Word o£ the availability of the KIT was spread through several 
mechanisms. In April, 1970, Dr. Stephen Klein presented a summary of 
the plans for the KIT to a meeting of the National Association of Ele- 
mentary School Principals in Philadelphia. Many of the principals and 
superintendents in attendance consequently wrote to express interest in 
learning more about the KIT or in becoming part of the field-testing 
program. These letters were responded to with the notification that 
interested schools and districts would be put on a list of schools to 
be considered in the national field testing. At the same time. Evalu- 
ation Comment and numerous technical research reports emanating from 
the Center referred to the KIT. These references brought additional 
interested responses which were handled in a similar manner. The Net- 
work of Schools, a network established by the lOX project when it was 
part of the Center, also expressed interest in maintaining a relation- 
ship with the Center through field testing of the KIT. It is iirportant 
to keep in mind that all of these 94 schools and districts mentioned 
above, the majority of the population from which the national sanple 
was chosen, voluntarily joined the field testing and had positive in- 
terest in it. 

Early in Fall of 1971, the schools and districts that had previ- 
ously expressed interest were contacted with an offer to become a part 
of the field-test sanple, i£ they qualified. The schools were to com- 
plete a questionnaire that would give the Center the information need- 
ed to gain representativeness. When the questionnaires were returned 
with most of the schools still expressing strong interest in participat- 
ing in the field tesing, it Wcis noted that several geographic areas 
were not represented in the population. These, areas were the south- 
eastern, the south central, and the northwestern regions of the coun- 
try. In telephone calls and follow-up letters |; to sister U.S.O.E . Re- 
gional Laboratories, commitments to aid in the;; recruitment of addition- 
al applicants were made by Dr. L. D. Fish of the Northwest Regional 
Educational Laboratory, Dr. K. W. Tidwell of fhe Soutiieastera.E^ 
tional Laboratory, and Dr. J, L. Olivero of the Southwestern Cooperative 
Educational Laboratory. In addition, similar, requests were made of Dr. 
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R. L. Bright at Baylor University, and Dr. S. S. Youngerman, Jr. , of 
the Boise (Idaho) School District. From these contacts, a number of 
schools and districts responded to help gain greater geographic repre- 
sentativeness. 

The population of schools at this point in time numbered 108, with 
some heaA^ concentration in the Mid-Atlantic region (New York, New Jer- 
sey, Pennsylvania, and metropolitan Connecticut) and in Illinois. 

Final selection of a sample to number 79 was then made on the basis of 
geographical area, racial-ethnic conposition of students, and socio- 
economic level of school neighborhood. Six school districts in New 
York, New Jersey, Pennsylvania, Connecticut, and Illinois had account- 
ed for 42 schools applying for admission to the field testing. These 
schools were predominantly in higher socio-economic level neighborhoods 
(professionals and white collar workers) and most of the schools con- 
tained a student population of over 90 percent white students. Twenty- 
nine of these schools were eliminated from the sanple. The remaining 
13 schools represented all six districts and allowed for maximum diver- 
sification in socio-economic status of neighborhood served and student 
racial-ethnic mix. 

The California Sampling 

The California sanple was arranged through the California Elementary 
School Administrators' Association (now a part of the Association of 
California School Administrators), through which much of the initial 
pilot testing had been arranged. Dr. Edward W. Beaiibier, Director of 
the Association, proposed a statewide san^le of schools that would be a 
part of the Association's evaluation conponent . Dr. Beaubier further 
proposed that the Association enploy various techniques in the inple- 
mentation of the KIT. Specifically, the Association was to "ride herd" 
on the implementation , and would install evaluation leaders for s atel- 
lite schools throughout the state. The novel approach to this uiple- 
mentation promised considerable additional information on how the KIT 
could be enployed, and so the Center agreed to include 100 schools (the 
actual number included was 103) in a California sanple. These schools 
were selected by Dr. Beaubier from members of his evaluation coiiponent. 
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Hie schools o£ the evaluation con^onent contracted with CESAA and paid 
$1000 per year for inclusion as evaluation con^onent schools (the Center 
received no portion of this payment) . This detracted from the diver- 
sity of the con5)onent schools (only schools who could afford the "fee" 
could be involved) and this was reflected in the California field-test 
sanq)le for the KIT which was chosen by Dr. Beaubier from the con^onent 
schools. 

Whereas the distribution o'£ the booklets of the KIT for the nation- 
al sanple was handled by the Center staff, Dr. Beaubier' s staff handled 
the actual distribution of materials to the California sanple schools. 

He also served as a "middleman" in the collection of relevant KIT data 
(returning questionnaires, tally sheets), forwarding such materials to 
the Center for evaluation and analysis. The procedui'e used in the dis- 
tribution of booklets and accon^janying accessory materials and the ques- 
tionnaires to be returned involved sending the necessary materials and 
booklet to the participating school and upon receipt of the coii5)leted 
questionnaire for that booklet, to send the next booklet with its acces- 
sories (if any). 

Ihe Case- Study Field Testing 

A third strategy in field testing of the KIT was an intensive case 
study of its inplementation in several schools. Discussion of this ap- 
proach is delayed for a later section, as the procedures and methods 
were considerably different from the two field-test approaches described 
above. 

Description of the National Sample 

The average starting grade taught in the schools in the national 
sanple (with kindergarten equal to zero) was .61, the average highest 
grade taught was 5.97. Fifty- four percent of the schools were K-6 
schools, eleven percent were K- 8, 20 percent were K- 5, ^d 15 percent 
used other grade organization patteriis.'^; The mean approximate piq)il 
enrollment (September, 1970) was 492.5 with a standard deviation of 
43. 2 piq)ils . Thirteen percent of the schools had a pi^il population 
of 300 or less , 61 percent ranged betweai 301 and 600 , and 26'; percent 
had an enrollment greater than 600. V 
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Appendix A exhibits the school characteristics questionnaire from 
which demographic data were collected. The neighborhoods served by the 
schools were described by the principals as follows: 

Rural area 12 percent 

Residential suburb 42 percent 

Industrial suburb 1 percent 

Small town (5,000 or less) 12 percent 

City of 5,000 to 50,000 16 percent 

Residential area of a large city (50,000+) 13 percent 

Inner part of a large city (50,000+) 3 percent 

The mean racial-ethnic breakdown of the student body of all 
schools was: 



American Indian 

Mexican- Ameri can 

Negro 

Oriental 

Puerto Rican 

White 

"Other" 



2.96 percent 
2.12 percent 
6.68 percent 
.14 percent 
.14 percent 
87.59 percent 
.36 percent 



Four percent of the schools had an American- Indian student population 
of greater than 20 percent, 9 percent were more than 30 percent black, 
67 percent were more than 90 percent white, and 20 percent had a mixed 
distribution (such as 80 percent white, 20 percent minority) . 



The mean socio-economic composition in the neighborhoods served 
by the sanple was: 



Professionals and managers 
White collar workers 
Skilled workers 
Unskilled workers 



22.90 percent 

29.06 percent 
31.12 percent 

17.07 percent 



Twenty- four percent of the sanple had a conpositioi.vwith more than 25 
percent professionals and managers, 18 percent contained more than 40 
percent white collar workers, 23 percent were more than 40 percent blue 
collar workers, 14 percent contained more than 25 percent unskilled 
workers, and 21 percent could be described as "mixed" (not falling into 
the previous categories) in distribution. 

Twenty-nine percent of the sanple was from the Northeastern region 
of the country, 26 percent from the Southeast, 16 percent from the North 
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Central region, 13 percent from the South Central region, 13 percent from the 
Rocky Mountain region, and 3 percent from the Pacific region. 

Description of the California Sample . 

The average starting grade taught in the schools in the California sample 
was .33, the average highest grade taught was 6.22. Sixty-four percent of the 
schools were K-6 schools, 12 percent were K-8, 8 percent were K-5i and 16 per- 
cent had other organizational patterns. The mean approximate enrollment • (Sep- 
tember, 1970) was 605.37 pupils with a standard deviation of 58.1 pupils. Three 
percent of the schools had a pupil population of 300 or less, 49 percent ranged 
between 301 and 600, and 48 percent had an enrollment greater than 600. 

The neighborhood served by each school was described as: 



Rural area 7 percent 
Residential suburb 41 percent 
Industrial suburb 0 percent 
Small town (5,000 or less) 9 percent 
City of 5,000 to 50,000 13 percent 
Residential area of a large city (50,000 +) 27 percent 
Inner part of a large city (50,000 +) 3 percent 



The mean racial-ethnic breakdown of the student body of all schools was: 



American Indian 

Mexican- American 

Negro 

Oriental 

Puerto Rican 

White 

Other 



.37 percent 
12.79 percent 
3.98 percent 
1.53 percent 
.09 percent 
80 . 41 percent 
.10 percent 

Sixteen percent of the schools had a Mexican- American student population of 
greater than 30 percent, 4 percent were more than 30 percent black, 58 percent 
were more than 90 percent white, and 22 percent had a mixed distribution. 

The mean socio-economic composition in the neighborhoods served for the 
san5)le was: 

Professionals and managers 25.59 percent 

White collar workers 28.22 percent 

Skilled workers 29.21 percent 

Unskilled workers 16.98 percent 

Tu^enty-nine percent of the sanple had a coirposition with more; than 25 percent 

professionals and managers, 22 percent contained more than 40; percent white. 

collar workers, 26 percent were more than 40 percent blue collar workers, 21 

percent contained more than 25 percent unskilled workers , and 2 percent could 

be described as "mixed" in distribution. ^ 



o 

ERIC 



• r . 






•: 

T 









A. 



-16- 



RESULTS C; THE LARGE-SCALE FIELD TESTS 

National Sample- -Results of Questionnaire #1 . 

Ten schools in the sample of 79 schools did not return Questionnaire #1. 

Six of these schools were in three of the districts in which the Center had 
selected only some of the schools which the district had offered for participa- 
tion in the field testing. Another school was given to understand by a higher 
level in its district's hierarchy of administration that the field test would 
not take place until next year (a misunderstanding which was not clarified until 
it was too late for the school to participate in the field testing) . Hie eighth 
school sent a letter of apology stating that circumstances within the school and 
the district made it impossible for them to implement the KIT this year (the 
circumstances were not specified) . The ninth and tenth schools did not reply 
to the Center's repeated efforts (by letters and postcards) to ascertain the 
state of the KIT's implementation. One of these two schools which ceased to 
participate in the field test was a last-minute entry to the field testing. The 
sanple which completed and returned Questionnaire #1 consisted of 69 schools from 
all parts of the country (an 87.3 percent return). Questionnaire #1 can be 
found as Appendix B. 

The responses of the sample to planning and decision-making procedures in 
the school relative to the instructional program, budget, and teacher assignment 
were as follows (percent of sample responding affirmatively in each cell in 
the matrix) : 
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planning is done 
at school level ; 
decision to imple- 
ment is made at 
school level. 


planning is done 
at school level; 
decision to imple- 
ment is made at 
district level. 


plans and 
decisions 
are made 
at district 
level . 


Instructional 

program 


45% 


. -V 58% 


23% 


Budget 


;/ ■ m ■ 




r22%. 


Teacher 

assignment 
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The majority of schools responded that the planning is done at the school level 
and the decision to impiemont is made at the district level in all three areas 
of concern: instructional programs, budget, and teacher assignment. Almost 

half of the districts reported that the instructional program planning and 
the decision to implement it were made at the school level. Budgeting and 
teacher assignment showed greater degrees of district control than instructional 
program processes. 

The following matrix relates to the same three areas as above. It indi 
who mthin the school is involved in the processes of planning and/or decision 
making when that process occurs at the school level (percent of sample respond 
ing affirmatively in each cell in the matrix). 

Planning Decision-making 







Prin. 


teachers 


parents 


Prin. 


teacliers 


parent3 


Instructional 

program 


91% 


93% 


35% 


81% 


71% 


9% 


Budget 


83% 


75% 


4% 


65% 


26% 


0% 


Teacher 

assignment 


86% 


46% 


1% 


70% 


16% 


3%, 



In most schools, teachers are involved in planning of the instructional program. t 

• • • * 

In many schools they are involved in decisions to implement the instructional \ 

program and planning the budget. A lesser number of schools involve them in 
planning teacher assignment. Teachers are much less involved in ,^he decision- 
making processes in schools than in the planning processes. Parents are only 
involved in one-third of the schools in the planning of the instructional program, 
and rarely in any other facet of planning or decision-making in the school a,reas 
of. concern.'' 

Thirty- four percent of the principals responded that their immediate superior . 
was the superintendent of schools, 2 Q-percent that he ■^s;- tlie eiementai^^ 

28 percent that he was the assistant , division, or deputy superintends . 

percent used other titles for their immediate superiors, ; . 
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Using a scale of 1 for most pressure to 4 for a blank or little pressure, the 
top three groups that place the most pressure upon the principal to institute 
educational reforms and/or new programs were: district administrators, 1.72; 

teacher organizations, 2.72; and parent -teacher associations, 3.14. Eighty -nine 
percent of the respondents indicated considerable pressure from district adminis- 
trators, 68 percent from teacher organizations, and 56 percent from parent- 
teacher associations. Other groups indicated, and their pressure scores, were: 
other professional groups, 3.52; student groups, 3.74; business organizations, 
3.90; women’s groups, 3.90; property owners association, 3.90; church or reli- 
gious groups, 3.91; veteran’s organizations, 3.96. 



Most principals responded that less than half of the families of their . 
students are represented at a typical meeting of the PTA or similar parent 
groups: 4 percent indicated that they had no parents’ organization, 28 per- 

cent that only a few families were represented at meetings, 41 percent that 
less than half are represented, 17 percent that about half are present, 7 per- 
cent that over half show up, and 3 percent that almost all of them are there. 

The normal procedures for learning of the attitudes of the principal’s 
teaching staff toward a particular issue regarding the instructional program 
(change in schedule , new materials , etc.) were to call a meeting of teachers 
(by 91 percent of the principals) and to contact the teachers individually 
(84 percent of the principals) . Only 10 percent contact teacher organization 
representatives . 



The normal procedures for the principals,: to learn of their school community’s 
attitude toward a particular issue regarding 'the school ’s instructional program 
were bringing it up at a meeting of the I^A or similar parent organization (noted 
by 64 percent of the principals) , checking with teachers (65 percent of the 
principals) and contacting a few parents (62 perce principals) . Only 

4 percent of the principals indicated that the occasion does not arise where 
■'^any of this.- is 'necessary;"’ ,v . ^ 

In the past two years 88 percent of the schools have inplemented at leas^ 



/one eixpefimental dr iim(nrative: project; 17 perce^ 

only one project: in vhich the desire vitOv i^ the 

|schbol itself . vTVehty-i^^ 

; . three pr^^ ectsv id perc^ four? projects ^ 9 percent fiv^ and' 12^ percent" 

ects. : The meah:Was 21;^^ 



ERIC 



Vf 








-19- 

o£ time 30 percent of the schools have implonented one such project where 
the implementation was initiated and desired by the school district. Eleven 
percent implemented two such projects, two percent implemented four projects, 
and 56 percent had no such projects. The mean was 0.6 projects per school. Few 
(11 percent) o£ these schools had any form of parental participation and fewer 
(2 percent) had pupil participation in the initiation or origination of projects. 
Eighty- two percent of the schools had teacher participation in the initiation or 
origination of .the projects, and in 64 percent of the schools the projects were 
initiated or originated by the principal himself. 

Using a scale of 1 for most important to 5 for least important, the prin- 
cipals ranked in order of importance the considerations relative to deciding 
on the allocation of funds for the next year's programs in their schools. Stu- 
dent needs as observed by teachers received a mean rating of 1.93 and 'was most 
important, followed by local board o£ education mandates with a mean rating of 
2.39. State mandates received a 3.48, student needs as indicated by standard- 
ized tests a 3.51, and suggestions from parents were deemed least important 
with a mean rating of 4.49. 

If the principal were suddenly given $500 to spend on any one instructional 
area in his school, he would decide how to spend the money in most schools equally 
on the basis of his own felt desires and in consultation with his teachers. 

Only a very few principals would involve either parents or pupils in this 
matter, and most would not take up this question with the district office. 




Principals were asked to list by name the standardized tests administered 
last year to first graders, third graders, fifth graders, and sixth graders. 
The tests most frequently listed by principals for the first grade were: 

Metropolitan Readiness Test 38% 

Stanford Achievement Test 15% 

Primary Mental Abilities 9% 

On the third grade level they were: ; 

Stanford Achievement Test 33% 

Iowa Tests of Basic Skills 20% 

Inrge -Thorndike^ Tests 12% 
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On the fifth grade level they were: 

Stanford Achievement Test 32% 

Iowa Tests, of Basic Skills 24% 

Lor ge -Thorndike Intelligence Tests 14% 

California Test of Mental Maturity 11% 

Metropolitan Achievement Test 10% 

On the sixth grade level they were: 

Stanford Achievement Test 37% 

Iowa Tests of Basic Skills 18% 

Lorge -Thorndike Intelligence Test 9% 

The pattern of test "popularity" or preference for the most frequently cited 
tests does not change from grades three to six. 

The rationale used for selection of the majority of the tests used in the 
first grade was: they were district mandated (45 percent of the principals 

checked this choice) ; the test measures an area desired to be measured (39 
percent) ; this is what the school has always done (7 percent) ; the price is 
right (1 percent); this is what other schools are doing (1 percent) . The 
rationale for test selection for the third grade was: they were district 

mandated (42 percent); the test measures an area desired to be measured 
(36 percent); this is what the school has always done (10 percent) ; they were 
state mandated (7 percent). The rationale for fifth grade was: they were 

district mandated (44 percent) ; the test measures an area desired to be measur- 
ed (35 percent); this is what the school has always done (10 percent) ; they 
were state mandated (1 percent); the price is right (1 percent). The rationale 
for the sixth grade was: they were district mandated (45 percent); the test 

measures an area desired to be measured (30 percent); this is what the school 
has always done (9 percent) ; they were state m^dated (4 percent) . The pattern 
to the rationale is extremely similar in all four grade levels. District 
mandating of a test is the most frequent reason for using a particular test. 
This is followed by a feeling that the test measures what is desireql to be 
measured and that this is what -the school has always ^^d^^ 

The primary use (application) for the scores from the tests bn any of the 
four grade levels was in student counseling and diagnosis (indicated, as a use 
in 68 percent of the schools averaged over all four grade levels^. Tlie next 
most frequently indicated applications for the scores (means over all four 
grade levels) were for pupil grouping (54 percent) and for staff curriculum- 
planning sessions (45 percent) . The scores were used in comparisons among 
O' - ::' 
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schools in 23 percent o£ the schools, for. teacher evaluation in 16 percent 
of the schools, and in pupil grading in 6 percent o£ the schools. 

The principal was asked to indicate what he thought the attitudes of 
teachers, guidance personnel, pupils, and principals were toward the standar- 
ized testing program in his school. The following matrix indicates the princi- 
pals's responses in percentage o£ total sample principals indicating a partic- 
ular cell of the matrix as an accurate assessment of attitude. 



Attitude 


teachers 


guidance 

personnel 


pupils 


principals 


provides necessary 
information 


46% 


62% 


4% 


57% 


it is required that 
the tests be given 


39% 


7% 


62% 


20% 


good measiure of pupil 
progress 


28% 


14% 


13% 


28% 


good measure of curriculum 
success 


■22% 


9% 


3% 


22% '' 


a good way of judging 
teacher competence 


1% 


1% 




6% 


forces teacher to 
teach to test 


4% 


1% 




6% . 


a waste of time; 
is harmful 


9% 


0% 


... 3% 


4% 



A majority of the principals felt that guid^ce personnel 'and principals perceive 
standardized tests as providing necessary information. Almost half of the , 
principal s felt that teachers also believed this. A ma j ority of the principals 
felt that pupils perceive the tests as something that just had to be given . 
Principals believed that the other main attitudes teachers and principals hold 
toward standardized tests are that they are required to be given and are a 



good measure of pupil progress and a good measure of curriculum success. (The 
patterns believed by teachers and principals was c|uite similar.) Few principals 
felt that many believed that standardized tests were a good way of judging 
teacher competence, force a teacher to teach to a test, or are a waste of time 
and are harmful. 

Itfhen asked to elaborate on their own attitude toward the use of standard- 
ized tests in their schools, 22 percent of the principals indicated that they were 
useful for student assessment and diagnosis, 18 percent said that they can 
indicate curriculum strengths and weaknesses but should not control instruction, 

18 percent felt that tests are not optimally used due to staff limitations, and 
13 percent said that they were only one source of pupil and program assessment. 

Principals tended to favor standardized tests as recognizable educational 
instruments but seemed to withhold great enthusiasm and were cautious of placing 
too much faith in standardized tests. "When asked to explain the hypothetical 
phenomenon that the average score on a standardized math achievement test given 
to all their third graders was extremely low in comparison vto what they had 
expected, most (84 percent) of the principals indicated that they would have to 
examine the test with their third-grade teachers before they could explain this 
result, 12 percent indicated that the test didn't measure what was taught, 1 
percent indicated that the result indicated a shift in the school community to 
a lovrer socio-economic status, and 7 percent believed other explanations were 
important . 

"When asked to specify the statement which best reflected their own attitude 
toward national norms on standardized tests, 25 percent indicated that they were 
good only for rough comparisons and guidelines. Eighteen percent felt that state, 
local . school, or classroom norms were better, and fourteen percent believed that 
they were not helpful or relevant. Twelve percent a.sked what national norms are, ■ 
9 percent believed that they were valuable but: not infall^^i^^^ 7 percent indica- ;i 
ted that national norms are always better than state or local norms for making 
decisions relative to the instructional program, 6 percent said that national noniis 
are always more reliable, 6 percent believed that they make schools look too 
good (or bad) , and 1 percent indicated that the advantage^ of national norms is 
that they are based solely on schools like their own. | 
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Califomia Sample- -Results of Questionnaire #1 . 

There were 12 schools in the California san^le that were sent Booklet I 
but never returned the questionnaire, even after being sent reminders of their 
negligence. With thej cooperation of the CESAA Project Evaluation Office it 
was possible to determine the reasons vhy these 12 schools were such early 
drop-outs. 

Unknown to us, five of these schools had chosen to implement another 
component of the CESAA Project Evaluation model instead of the KIT. It was 
thought at the beginning of the field testing that the 103 schools in the 
California sample had already chosen to field test the KIT, but apparently this 
was not the case. 



Another five of the schools never returned the questionnaire for Booklet I 
because they had dropped out of the CESAA Project Evaluation. Since the members 
of the Project had to pay to belong, it is not surprising that a few schools 
dropped out completely, probably due to a lack of funds. Of the remaining two 
schools, no information could be obtained regarding their reasons for dropping 
out. It is suspected that in one of the schools there was a 'change in princi- 
pals during the year, but this was not confirmed. The sample which completed 
and returned Questionnaire #1 consisted of 91 schools from t!he state (an 88.4 
percent return.) 1i / . | 

The responses of the sample to planning and decision -jiiaking procedures in 
the school relative to the instructional program, budget, |and teacher assign- 
ment were as follows (percent of sample responding affirmatively in each cell 
in the matrix) : ■ ' ,, : , \ ;i J; ■■ . ;:i;: 





planning is done 
at school level; 
decision to incle- 
ment is made at 
school level. 


planning is done 
at school level ; 
decision to incle- 
ment is made at 
district level. 


plans and 
decisions 
are made 
at district 
' level . 1 


Instructional 






■ ■ ■ .. 


program 




" ' - ^ " : •••. • , ■ ■' 


'■ ■ ; ■■ . • ■■ ' ■ 


Budget 






m 


Teacher y 


341 






assignment 
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The majority of schools responded that the planning and the decision to imple- 
ment are done at the school level in the area of instructional programs. Almost 
half reported that plans and decisions on budget were made at the district level 
and that the planning was done at the school 3.evel and the decision to implement 
was done at the district level in the area of teacher assignments and some in- 
structional programs. Budgeting and teacher assignment showed greater degrees 
of district control than instructional program processes. 

The following matrix relates to the same three areas as above. It indicates 
who within the school is involved in the processes of planning and/or decision 
making when the process occurs at the school level (percent of sample responding 
affirmatively in each cell in the matrix) . 

Planning Decision-making 






Prin. 


teachers 


parents 


Prin. 


teachers 


parents 


Instructional 

program 


97% 


97% 


35% 


88% 


79% 


11% 


Budget 


70% 


66% 


5% 


65% 


47% 


1% 


Teacher 

assignment 


79% 


55% 


2% 


73% 


34% 


1% 



In most schools y teachers are involved in planning of the instructional prpgr«iin. 

In many schools they are involved in decisions 'to implement the instructional 
progrsin, and in 3 . lesser number in planning the budget and teacher assignment.;^: : 
They are considerably less involved in the decision-making processes than in the 
planning processes. Parents are only involved in one.- third of the schools in , 
the planning of the instructional program, and rarely in any other facet of plan- 

ning or decision making in tihe school areas of concern ^ r 

FortyTthree. percent of the: principals responded that their imme 
was the superintendent of schools, 34 percent that he was the assistant , associate, 
or deputy superintendent, 15 percent that he was called the director of ©lenient ary 
education, and 8 percent used other titles for their immediate superiors, u . 

I KJC 

^ ^ 
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Using a scale o£ 1 for most pressure to 4 for a blank or little pressure, 
the top three groups that place the most pressure upon the principal to insti- 
tute educational reforms and/or new programs were: district administrators, 

1.64; teacher organizations, 2.52; and parent -teacher organizations , 3.05. 

Ninety percent of the respondents indicated considerable pressure from district 
administrators, 75 percent from teacher organizations, and 63 percent from par- 
ent-teacher associations. Other groups indicated and their pressure scores 
were: student groups, 3.52; other professional groups, 3.69; property owners* 

association, 3.79; church or religious groups, 3.85; women’s groups, 3.88; 
veteran's organizations, 3.89; business organizations, 3.90. 

^fost principals responded that less than half of the families of their stu- 
dents are represented at a typical meeting of the PTA or similar parent groups: 

2 percent indicated that they had no parents' organization, 27 percent that only 
a few families were represented at meetings, 51 percent that less than half are ■ 
represented, 7 percent that about half are present, 11 percent that over half 
show 1 : 5 ), and 1 percent that almost all o£ them are there. 

The normal procedures for learning of the attitudes of the principal's 
teaching staff toward a particular issue regarding the instructional program 
(cliange in schedule, new materials, etc.) were to call a meeting of teachers 
(by 91 percent of the principals) and to contact the teachers individually (87 
percent of the principals) . Only 11 percent contact teacher organization repre- 
sentatives. 

The normal procedures for the principals to learn of their school conmiuni- 
ty's attitude toward a particular issue regarding the school's instructional pro 
gram were bringing it up at a meeting of the PTA or similar parent organization 
(by 78 percent of the principals) , /contacting a few parents (74 percent of the 
principals) , arid checking mth teachers (64 percent of the principals) . Only 4 
percent of the principals indicated the pccas ion does riot arise where any 
of this is necessary. ' 

In the past two years 18 percent; of the schools have implemented one exiier- 
imental or innovative -project in which the desire to implement had been initiat- 
:: ed by the school itself. Twenty-four percent implemented two such projects , 23 
percent three projects , 11 percent four projects , 12 percent five pro j ects, and 
12 percent had no such projects . The mean was 2.4 projects per school; In the 
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same period o£ time 21 percent o£ the schools have implemented one such pro- 
ject where the implementation was initiated and desired by the school district. 
Ten percent implemented two such projects and 8 percent implemented three pro- 
jects, while 61 percent had no such projects. The mean was 0.6 projects per 
school. Few (14 percent) o£ these schools had pupil participation and fewer 
(12 percent) had any £orm o£ parental participation in the initiation and orig- 
ination o£ the project. Seventy-seven percent o£ the schools had teacher par- 
ticipation in the initiation or origination o£ the proj ects , and in 67 percent 
o£ the schools the projects were initiated or originated by the principal him- 
sel£. 

Using a scale o£ 1 £or most important to 5 £or least important the princi- 
pals ranked in order o£ importance the considerations relative to deciding on 
the allocation o£ £unds £or the next year's programs in their schools. Student 
needs as observed by teachers received a mean rating o£ 2.14 and was the most 
important £actor , £ollowed by local Board o£ Education mandates with a mean 
rating o£ 2.46. Student needs as indicated by standardized tests received a 
2.98, state mandates a 3.41, and suggestions £rom parents were deemed as least 
important with a mean rating o£ 4.20 

I£ the principal were suddenly given $500 to spend on any one instruction- 
al area in his school he would decide how to spend the money in most schools 
primarily on the basis o£ consultation with his teachers and secondarily on the 
basis o£ his own £elt desires. Less than a £i£th o£ the principals would in- 
volve either parents or pupils in this matter, and most would not take up this 
question with the district o££ice. 

Principals were asked to list by name the standardized tests administered 
last year to £irst graders, third graders, £i£th graders, and sixth graders . 

The most £requently listed tests £or the £irst grade weye: 

Cooperative Primary Reading Test 78% 

Harper-Row Reading Readiness Tests 16% 

Metropolitan Reading Readiness Test 13% 

On the third grade level they were: 

Stan£ord Reading Test 47% 

Stanford Achievement Test 37% 

California Test of Mental Maturity 13%^^^^ 

Harper-Row Reading AchievOTeht'Test 11% 

Lofge-jThorndike Intelligence Test 10% 
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On the fifth grade level they were: 



Comprehensive Tests of Basic Skills 19% 
Stanford Achievement Test 12% 




California Achievem 




On the sixth grade -l-evel'Tihey were: 



Comprehensive Tests of Basic Skills 73% 
Lorge-Thorndike Intelligence Tests 69% 
Stanford Achievement Test 12% 



The pattern of test "popularity" or preference for the most frequently cited 
test changed over the grade levels , due to different state mandates . 

The rationale used for selection of the majority of the tests used in the 
first grade was: they were state mandated (80 percent of the principals 

checked this choice); they were district mandated (23 percent); the test meas- 
ures an area desired to be measured (20 percent); this is what other schools 
are doing (1 percent) ; and this is what the school has always done (1 percent). 
The rationale for test selection for the third grade was: they were state man- 

dated (77 percent); they were district mandated (32 percent); the test meas- 
ures an area desired to be measured (23 percent) ; .this is what the school has 
always done (3 percent) ; and this is what other schools are doing ( 2 percent) . 
The rationale for the fifth grade was: they were district mandated (30 percent); 

the test measures an area desired to be measured (25 percent); the tests were 
state mandated (16 percent); this is what the school has always done 1(4 percent); 
the price is right (1 percent) ; and this is what other schools are doing (1 per- 
cent). The rationale for the sixth grade was: they were state mandated (7o per- 

cent); they were district mandated (29 percent) ; tJie test measures an area de- 
sired to be measured (16 percent) ; this is what the schooT has always done (3 
percent) ; and this is what other s'dibols are doing (3 percent) . The pattern to 
the rationale of test selection is alike in grades one, six as might 
be expected from the state mandated testing system in California for these 
grades . State mandating of a test is by far,the most frequent reason for 
using a particular test. This is followed by district mandate and the feeling 
that the test measures what is desired to be measured. In grade five the patteni 
differs with much less stress on state mandates , arid more on selecting a test 
which measures an area desired to be measured. 
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The use (application) of the scores from the te?tT^cre- 3 ~iv 24 _d.^^^ very 
much for grades one, three and six. The primary applications (mean perceht- 
ages for all three grade levels) were in student counseling and diagnosis 
(indicated as a use in 69 percent of the schools) , pupil grouping (61 per- 
cent), and staff curriculum-planning sessions (S3 percent).- The scores were 
used in comparisons among schools in 34 percent of the schools, for teacher 
evaluation in 18 percent of the schools, and in pupil grading in 4 percent 
of the schools. The pattern of use did not differ much in the fifth grade 
though the percentage of the schools reporting a particular use did. The 
primary uses were in student counseling and diagnosis (S3 percent), pupil 
grouping (43 percent) and staff curriculum-planning sessions (42 percent). 
These were followed by comparisons among schools (15 percent), teacher 
evaluation (12 percent) , and pupil grading (4 percent) . 



The principal was asked to indicate what he thought the attitudes 
of teachers, guidance personnel, pupils, and principals were toward the 
standardized testing program in his school. The following matrix indicates 
the principal's responses in percentage of total sample principals indi- 



cating a particular cell of the 



matrix as an accurate 



assessment of attitude. 



Attitude 


teachers 


guidance 

personnel 


pupils 


principals 


provides necessary 
information 


45% 


52% 


9% . 


51% 


it is required that 
the tests be given 


48% 


22% 


68% • 


38% 


good measure of pupil 
progress 


20% 


13% 


10% 


15% ■ 


good measure of curriculum 
success 


4% 


7% 


■ 1% ■ V-'- 


’ 14% 


a good way of judging 
teacher competence 


• 2% 


0% 


0%. 


3% 


forces teacher . to ^ 

teach to test 


■ 4% '■■■: 


3% 


1% 


V 7% 


a waste of .time; 
is harmful 


'it ' ' • 




' 3% ./ • 


5% 



A majority of the principals felt that guidance personnel and principals 
perceive standardizer^. tests as providing necessaiy information^ half 

the principals felt: that teachers also believed this. the 
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principals believed that pupils looked upon the test as something that 
just had to be given. Almost half of the principals indicated that teachers 
felt this way, and over one- third indicated that principals maintained this 
view also. A fifth of the principals noted that teachers believed standard - 
4-^gd.J-^g.t.g — to he gnod -nieasa«:^--s^-pap^^ and that counselors believed 

that the tests were required to be given. Few principals felt that many saw 
standardized tests as a good way of judging teacher competence, force 
a teacher to teach to a test, or are a waste of time (and harmful). 



When asked to elaborate on their own attitude toward the use of stand- 
ardized tests in their school, 27 percent indicated that they are not valuable, 
arc duplicative, of no help or irrelevant. Twenty-five percent felt that 
the tests were useful for student assessment and diagnosis, 19 percent expressed 
generally positive attitudes, 18 percent believed that these tests can indi- 
cate curriculum strengths and weaknesses but should not control instruction, 
and 9 percent indicated that the tests are only one source of piq)il and pro- 
gram assessment. 



Principals tended to favor standardized tests as recognizable educational 
instruments but seemed to withhpld great enthusiasm and to be cautious of 
placing too much faith in standardized tests. When asked to explain the hypo- 
thetical phenomenon that the average score on a standardized math achievement 
test given to all their third graders was extremely low in comparison to what 
they had expected, most (88 percent) of the principals indicated that they 
would have to examine the test with their third-grade teachers before they 
could explain this result, 11 percent indicated that the test didn’t measure 
what was taught, 2 percent said that they really didn't know how to explain 
it, and 7 percent believed other explanations were important 



When asked to specify the statement which: best reflected their own 
attitude toward national norms on staridardi zed tests , 32 percent asked what, 
national norms were.^^^:T^ that national norms . were not, 

helpful and were irrelevant, whereas 8 percent believed that tiiey were 
valuable but not infallible . Seven percent believed that state , » 

school , and classroom norms are better , 7 percent thought ,t^ 



norms were good only for rough comparisons and guidelines, ;4 percent indi- 
cated that national norms are always more reliable, and 1 percent said ■ 







l7..' 
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that national norms are always better than state or local norms for making 
decisions relative to the instructional program. 

National Sample - Results of Questio nnaire #2 

There were 79 schools in the original sample, of which 69 returned Ques- 
tionnaire #1 and 46 returned Questionnaire #2. This attrition caused minor 
changes in the geographic composition of the sample; suburban schools and 
North Central schools tended to persist and made up a larger proportion of 
the dwindling sample. Small cities and the Southeast and Rocky Mountain 
states became less represented in the sanple as schools in those areas ' , : 
dropped out. However, the attrition in the sample did not alter the racial, 

socio-economic, or geographic composition of the sauple in any signifi- 
cant way. 



Original Sample 
Percentage 



Rural area 
Residential suburb 
Industrial suburb 
Sma.ll town (5,000 or less) 

City of 5,000 to 50,000 
Residential area of a large 
city (50,000 +) 

Inner part of a large city (50,000 +) 



Respondents 
to Questionnaire #2 

Percentage 



12 

42 

1 

12 

16 

13 



11 

39 

0 

15 

11 

20 



American Indian 

Mexico- terican 

Negro 

Oriental 

Puerto Rican 

Wliite 

other 



less than 
less than 



less than 



3 

2 

7 

1 

1 

88 



less than 



less than 



4 

2 

6 

1 

0 

87 

1 



Profess ionals and managers 
White collar workers 
Skilled workers 
Unskilled workers 



23 

29 

31 

17 



21 

29 

32 

17 



Northeast 
Southeast 
North Central 
South Central 
Rocky Mountain 
Far West 






29^ 

26 

16 

13 

13 

3 



30 

17 

22 

^-M7-;' ■: 
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The second booklet of the KIT provides a system for parents, teachers, 
principals and others to decide which of the school's goals are of greatest 
importance. For this purpose a list of goals is provided and a step-by- 
step process of information - gathering is described. Questionnaire #2, 
dealing with this booklet, can be found in Appendix C. 

The school principals responding to Questionnaire #2 offered 25 sug- 
test ions for student performance goals to be added to the list provided 
by the KIT. Among the 25 are 20 different suggestions, including 9 which 
are nearly identical to goals already on the KIT 's list. Two other impor- 
tant suggestions (ecology and drug awareness) are subsumed in objectives 
already on the KIT'S list. They will, however, be explicitly incorporated 
into the goal descriptions. 

Most of the other suggestions are concerned with the affective do- 
main and may be subsumed under existing goals in the KIT's list. Words 
describing affect tend to be value-laden, imprecise, and poor tools for 
communication; causing implication and inference to fail to match| For 
example, a school principal may list "self-discipline" as a new goal, 
not realizing that it is part of goal 2A, Dependence - Independence. 
Another school principal may list "inquiring, humanistic approach" as a 
new goal; while we feel that this goal is too broad and vague, if it were 
better defined, it would fit into one or more of the categories on the 
KIT's list. Another principal may list "independence," feeling that his 
concept is not the same as our concept of Dependence - Independence; in 
this case we have both of the problems given above: semantic differences 

and different levels of generality. 

Sane of the suggestions in the affective domain were: 

self-discipline 

attitudes 

responsibility vs . duty 
rights vs. privileges 
dignity 
independence 

inquiring, humanistic approach, 
understands his mastery level 

There were two other suggestions. One school principal suggested 
typing and another suggested work experience. The list of goals provided 
by the KIT does not include provision for either of these suggested goals. 



Twenty suggestions were made for goals other than student performance 
goals. Several of these were staff development goals: 

staff participation in evaluation 
improved guidance skills 
improved evaluation skills 
improved personnel relations 
teacher training and self-esteem 

Many suggestions involved better communications with the community, with 
greater community involvement by teachers and greater school involvement 
by parents. One principal suggested as a goal, "Recapture true values," 
but did not define which values are true; and another principal suggested 
the goal of increasing use of the school plant. Finally, seven respondents 
volunteered their opposition to the use of goals which are not student - 
performance goals. These additional suggestions are outside the domain 
of student outputs, and therefore will not be incorporated into the KIT. 

Only six respondents felt that there was too much similarity among 
the goals. The greatest areas of concern were science (named by three 
respondents) and the concept of independent application of skills. 

Seventy- four percent of the respondents felt that the goals were or- 
ganized in a manner appropriate for their schools. Only one felt that 
the goals and the organization of goals are not appropriate for a wide 
variety of sdiools and types of students. Some criticism of the organi- 
zation of goals was expressed: that sane goals were too advanced for 

their schools, that the vocabulary of the cards was too difficult, or 
that the cards did not represent adequately unitary concepts. There 
were two objections to the organization of religious goals and one to 
the organization of foreign language goals. There was no objection 
to, or even mention of, the organization of affective goals. 

IVhen asked to indicate preference for change to more general goals 
or more specific goals, 46 percent called for more specific and numerous 
goals, while 41 percent called for more general and less numerous goals. 

Respondent schools v/ere asked to list all of the additional student- 
performance goals suggested by any person involved in the card-sort 
technique, including teachers, parents, school board members and/or others. 
There were 70 suggestions overall, including 45 different suggestions. 



Many of the suggested new goals are part of existing goals; for example, 
environmental effects (39B) , drug education (23B), foreign language com- 
prehension (11) , race understanding (40A) , library use (14A) , and logic (8) 
Many of the suggested goals (listed below) are in the affective area, 
and may be subordinate or superordinate categories of the existing 
list. 

enthusiasm 

cooperation 

humanistic approach 

free and whole individual 

enjoyment of school 

interpersonal relationships 

understanding beliefs of others 

feeling of acceptance 

self-control 

self-discipline 

personal goal setting and motivation 

efficient use of time 

obedience 

There are a number of suggestions which should be seriously considered as 
additions to the list: 

morals and ethics 

lifelong learning 

Trachtenberg mathanatics 

homenaking 

child care 

typing 

drama 

Among the suggested goals not dealing with student perfoimance were: 

individualize 
library freedan 

systematic reasoning in relationship 
between parent, child, and teacher 
community resource use 

character education: teach children to live in the 
present rather than always en^hasizing the future 

School principals felt that the system of rating goals was easy for 
themselves (89 percent) , easy for the teachers (93 percent) , and to a 
lesser extent easy for the parents (70 percent) . Several difficulties 
of the goal-rating process were named. Six principals said that parents 
had trouble understanding the cards and five principals noted that 
parents had trouble with the directions for sorting. There were five 
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references to excessive time-consumption for the card-sort process, 
and three respondents specifically referred to delays in getting 
parents to return the cards. Two principals would like to have more 
card sets to speed up the process. There were five references to 
difficulties in tallying or understanding the directions for tallying. 

Only one school principal indicated that he would prefer some 
other way of rating the goals; parents in his school cannot under- 
stand the cards, so he felt that an interview technique would be 
better. Another respondent liked the system, but would prefer the 
cards not be shuffled before the card-sort. The other 44 respon- 
dents had no recommendation for change in the goal-rating system. 

Most respondents (87 percent) felt that the goals which they 
identified as "most important" by this method would also be chosen 
with other methods. Two respondents disagreed, and five others were 
not sure. 

Most principals report that using this method changed their 
knowledge of how teachers and parents value educational goals. Hie 
following observations were noted by principals as being new or dif- 
ferent or stronger than they had thought would be the case: 



Observations 



H of Principal 



That teachers value attitudinal goals 
(higher than content goals) 

That parents value attitude above content 
That parents value content above attitude 
That teachers and principal agree 
That teachers and principal disagree 
That parents and principal agree 
That parents and principal disagree 
That teachers and parents agree 
That teachers and parents disagree 
That individuals in a single group vary widely in 
their rating of a given goal 
That individuals do not vary widely 
That ratings were low for music 
That ratings were low for foreign language 
That rating were low for art 
That rating were low for social studies 
That parents (except for a vocal minority) 
approve of sex education 



10 

8 

3 

4 

5 

5 
1 

6 
2 



1 



3 

3 

4 
2 
1 
1 






-35- 



Ninety-five percent of respondents found that the instructions for 
the collective viewpoints were understandable; 93 percent could follow 
the procedure, and 85% found the computations easy. Seventy-four per- 
cent felt the results were commensurate with the effort expended; those 
that felt that it was not worth the effort conplained not about the 
results, but about the lengthy computations without clerical assistance. 
One respondent suggested the use of IM scoring sheets, and two others 
asked for reports on the overall results of the study. 

Thirty-nine percent of respondents claimed that they tried to com- 
bine various viewpoints to get one set of goals. Three respondents took 
the ten top-rated goals fran each list and made judgments from that data, 
and one respondent used the Delphi technique. The others used an average 
or weighted average of combined data from all groups: teachers, parents 

and principal. 

Randan sampling of parents was attempted by 70 percent of the schools. 
Many of them (39 percent of the entire san 5 )le) noted problems in random 
sampling, although most of them found it easy (50 percent) and worth 
the effort (54 percent) . The greatest problan noted was getting returns 
from the parents. Two respondents noted specifically that parents had 
trouble understanding the method, and it may be assumed that this factor 
contributed to the poor rate of return of information in other schools. 

Stratified sampling was attempted by only 4 schools: three noted 

problans, one found it easy, and three felt it was worth-while. Problems 
mentioned were low returns, particularly from lower socio-economic groups; 
lack of a census of the caranunity and problems in selection procedure; and 
use of the random number chart. 

The general reaction of teachers to the process of goal rating, as 
reported by school principals, was favorable (80 percent) or mixed (10 per- 
cent) , with 10 percent of the principals making no conment about teacher 
reactions. No principals reported that the teachers had negative atti- 
tudes tovmrd the -process . 

Parent attitudes, as reported by school principals, were 70 percent 
favorable and 20 percent mixed; again 10 percent of the principals 
offered no report on parent attitudes. 



- 36 - 











ERIC 

MMiffilffTlTLiU 

w 






The overall reaction of the respondents to Booklet II was 75 percent 
favorable, 11 percent mixed, and 4 percent unfavorable; while 6 percent 
had no comment. Some of the negative reactions were: 

There are built-in prejudices, albeit small . 

Goals should be consolidated. 

Sampling methods are too sophisticated , 

Confusing only at first. 

Eighty- two percent of the respondents would recommend the procedures 
contained in Booklet II to other principals. Two percent (1 respondent) 
would not, because he fc... that it requires a "more basic background 
before there can be a real value,,," Six percent did not respond to this 
question and 8 percent indicated that they would reconmend the procedure 
with reservations , 

Additional conments were offered by many school principals . Some 
of these were laudator)^, and some reiterated points made in earlier com- 
ments. Other comments are given below: 

"I had hoped to find new directions for our 
school but instead, found, that our present areas 
of emphasis are those given high priorities by 
both parents and teachers. Maybe this is a good 
indication, but .couldnft.it also mean that what 
we are doing is right because we are doing it?" 

"Parents were all college trained people which 
migjitbe considered a stratified study, however, 
this .would be a typical sanpling of our community." 

"Couldn't figure out how to. use coral and 
yelloiv tally sheets." (note- -many respondents used 
the wrong sheets.) 

"Judging from niy past experience with parents 
the instructions on the green sheet given to parents 
should be siji^lified and different from the ones for 
teachers. The cover letter for parents (page 21) needs 
some additions, the exact wording of whi^ would de- 
pend on each principal's degree of rapport and personal 
contact with his community. Principals should be 
given-^ outline of the letter and instructed to 
adapt.it to their own situation," 

California Sample - Results of Questionnaire #2. 

Hie original sanple in California numbered 103 schools, of which 91 
returned the first questionnaire and 63 returned the second. The changes 

39 
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in the geographic, racial -ethnic and socioeconomic composition of the sample 
due to this attrition were quite minor: there were slight increases in the 
representation of small town and inner-city schools, minority students, and 
children of unskilled workers. 



Rural area 

Residential suburb 

Small Town (5,000 or less) 

City of 5,000 to. 50,000 
Residential area of a large city 
(50,000+) 

Inner part of a large city (50,000 +) 

American Indian 

Mexican-American 

Ne^o 

Oriental 

White 

Professionals and managers 
White-collar workers 
Skilled workers 
Unskilled workers 





Respondents to 


Original Sample 


Questionnaire #2. 


Percentage 


Percentage 


7 


5 


41 


40 


9 


11 


13 


14 


27 


25 


3 


5 


1 


1 


13 


14 


4 


5 


2 


2 


80 


78 


26 


25 


28 


27 


29 


29 


17 


19 


IS additions to the list of student- 



performance goals offered by the KIT . Many of them are restatonents or 
reorganizations of the existing goals , but many do not fit into the KIT s 
list. Suggestions worthy of note include: 



Oral language in dialect 
Student government 
Need for rest 
Use of basic hand tools 
Sense of humor 

Recognizes own strengths and weaknesses 

Develops own goals and means to achieve these goals 

Natural curiosity , . , . , 

Decision-making (thinking skills, generalized prob Ion- 

solving ability) 

Courtesy 

Ability to resolve disagreements 

Sensitivity to others, (empathy, understand^g of human 
differences, etc.) 



o 
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One principal suggested that "the child develop ability to adapt to 
different learning environments," rather than the converse of adapting 
learning environments to the child. Several principles indicated that 
the list was comprehensive enough, but one commented, "It is long enough." 

Respondents from California were much more prolific than those in the 
national sample in suggesting goals other than student -performance goals. 
The Californians were concerned with the problems mentioned by the national 
sample (community involvement in schools, teacher involvement in conmunity, 
communications among all groups) , but were also concerned with many other 
areas. Among the goals suggested were: 

Flexible physical plant 

Flexible scheduling 

Schools as teacher- training institutes 

Decentralized school organization 

School finance 

Federal aid 

Voucher system 

Teaser, commitment to. work in minority . schools 
Positive learning atmosphere, in Ae homes 
Student . pride in school 
Principal efficiency 

Principal delegation of responsibility to staff 

Staff _ acceptance of individual differences 

Control of students 

Participation 

Morale 

Competence 

Inter-relations 

Self-direction and motivation 

Attitudes to children 

Skill -in evaluating individual pupil progress 

Twenty- seven percent of the respondents found similarities in the KIT 
goals. They listed 54 different sets of similarities, involving almost 
every goal in one set or another. The greatest concentrations of similar 
goals were in the areas of Arts and Crafts; Health, Education and Safety; 
Geometry and Measurement; Foreign Language; and Science. 

California school principals were even more adamant than the national 
sample, that the number of goals be reduced. Seventy percent favored fewer 
and more general goals, vdiile 22 percent called for more specific and more 
numerous goals; 8 percent offered no opinion. There was considerable senti- 
ment that the wording on the cards was too difficult or ambiguous. 
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particular ly for parents . There were . other suggestions to improve the 
organization of goals: 

Do not shuffle cards 
Translate cards into Spanish 

Include more goals relevant to practical living experiences 

Principals were asked to list new student performance goals suggested 
by teachers or parents in their schools. Suggestions not previously listed 
in this report are given below: 

What to wear at proper times 
Use of leisure time 
Criticizes constructively 
Money and handling thereof 
Typing 

Exjxjsure to state and local laws 
Study -habits 
Religious tolerance. 

Ownership (?) 

Attitude toward welfare and military service 
Winning- fail ing 

Release emotional feelings freely 

Goals other tlian student -performance goals which were suggested by 
teachers or parents (not previously listed) were: 

Make education responsive to parents 
Disseminate knowledge of public and Catholic high schools 
in area, courses available, and cost. 

California principals found the system of rating goals to be easy 
for themselves (84 percent) and for teachers (82 percent) , but only 65 
percent of the principals report that the system was easy for parents . 
Various difficulties were cited specifically by the California principals . 
They were less concerned than was the national sample about the time- 
consuming tabulations, but they were much more concerned about the 
difficulty parents had had with the vocabulary of the cards. One princi- 
pal noted that the words "Ideational," "Aural," and "Spatial" were not 
generally understood. 

Other problems cited were: 

Finding teacher time 
Not enough cards 
Goals too sinple (1) 
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Some goals were inappropriate 

Goals interpreted differently by different groups 

Prefer process to output goals 

Seventeen percent of California respondents would prefer some other 
way of rating the goals. Some were merely reiterating their feeling that 
fewer goals should be used. One suggested a questionnaire rather than a 
card-sort; another suggested a tally-sheet format. One requested an out- 
line of the 145 goals to be given out before goal selection. Another 
felt that the process was "too traditional - where does individualization 
fit in?" Another principal objected to the criterion for determining 
importance; he suggested, "Use as a criteria 'those goals for which the 
school should accept full responsibility should be rated 5, etc.'" 

iVhen asked if other rating methods would yield a different set of 
"most important" goals, 14 percent responded "yes" and another 21 percent 
responded with a partial or conditional yes. Few seemed to feel that a 
different system would be superior; more often they commented on perceived 
flaws in other systems: 

"If goals had been selected without aid. of . Gcal Cards, 
more ..emphasis on academic areas as 'Most Important' 
may have resulted." 

"Yes. If choices were different, results would be 
different." 

"Only if principal influenced selections via his intro- 
ductory comments." 

"Yes. If a questionnaire were used, a 'pattern of 
responses' may develop, i.e., the rating of one item 
is related to sequence in ^ich they are typed." 

Most California principzils (84 percent) indicated that the rating 
method provided them with some useful information which confirmed their 
perceptions or taught them something about teachers ' and parents ' values . 
Ihis information was about the same for the California sanq)le as for the 
national san^le. All data given below are observations by the principals 
relative to their own previous perceptions. 
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Observations # of Principal 



That teachers value affective areas higher than 
expected I . ^ 

That teachers value cognitive areas higher 4 

That parents value affective areas higher 15 

That parents value cognitive areas higher 7 

That teachers and parents agree more than expected 7 

That -teachers and parents agree less than expected 0 

That differences in values among teachers are small 3 

That differences in values among teachers are large 5 

That ratings were low for art ^ 2 

That rating were low for sex education 1 

That parents in a parochial school given low importance 1 

to religion 



The principals indicated that the instructions for the card-sort 
were understandable for themselves (95 percent) and the procedure was 
easy to follow (92 percent) . The computation was easy for 78 percent 
and the results were commensurate with the effort expended for 71 per- 
cent. Those principals who felt that the effort was not worthwhile 
listed 18 reasons why: 

Tallying too time consuming 10 

Too many goals 4 

Need larger number of respondents, too hard 
to get them and do tallying 2 

Need less respondents 1 

Small print in format is hard to tabulate, 
too many items on one page 1 

Thirty-six respondents (57 percent of the sample) indicated that 
they had attempted to combine the viewpoints of various groups into one 
set of goals for the school. TVenty-five of these respondents specified 
a particular method of combinirg viewpoints: 

Average or weighted average 13 

Committee 4 

Comparing top ten for each group 4 

Raiddng 2 

Judgment 2 

Random sampling was done by 59 percent of the school principals. 

It was easy for 37 percent (of the entire sample) and considered worth- 
while by 48 percent (again, of the entire sample). Some of the pro- 
blems encountered were parent availability (cited by 5 principals) , 
getting returns from parents (2) , parent understanding of directions (2) , 
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too few subjects (2), lack of cooperation (2), and that only one 
socioeconomic group was represented (1) . One principal noted that 
parents were selected by the principal; he misimderstood either the 
question or the meaning of "random sampling." 

Stratified sampling was reported to have been done by 19 percent 
of the sample (12 schools), each of whom found it worthwhile and three- 
fourths of. whom found it easy. Three problems were noted by the res- 
pondents: time, high transience, and lack of stratification in the 

community. 

Principals were asked to indicate the overall reaction of teachers 
to involvement in Booklet II procedures . Sixty-eight percent of the 
principals reported favorable reactions; 22 percent reported mixed re- 
actions from teachers, and 3 percent reported that the teachers were 
threatened and frustrated by the procedures. Seven percent did not 
offer any indication of teacher reactions. 

Seventy-three percent of the principals reported favorable reac- 
tions from the parents, 19 percent indicated that parents had mixed 
reactions, and one principal noted only negative reactions fran the 
parents. Six percent of the principals did not offer any report on 
parent reactions to Booklet II. 

The principals reported their own reaction to Booklet II as 
favorable (73 percent), mixed (16 percent), or negative (2 percent: only 
one principal). Nine percent did not respond to this question. Seventy- 
seven percent of the principals acknowledged that they would recommend 
the procedures in Booklet II to other principals. Several noted that 
they .had already done so. 

There were several additional conments from principals in the ‘ 
California sample: 



National Sample - Results of Questionnaire ifZ 

Attrition in the national sample reduced the number of schools to 34. 



Include some suggestions on developing consensus 
Too many different colored tally sheets 
Need instruction index 
Need wider range of ratings 



ERIC 




V 



-43- 



I? 



S: 

I: 



lx 








The loss was greatest from the Northeastern states, and was least among 
children o£ unskilled workers. Otherwise the attrition was evenly dis- 
tributed and had little effect on the proportions of ethnic and socio- 
economic groups or regional and residential origins. 



Respondents to 

Original Sample Questionnaire If 3 , 



Percentage 



Percentage 



Rural area 12 

Residential suburb ^2 

Small Town (5,000 or less) 12 

City of 5,000 to 50,000 16 

Residential area of a large city 
(50,000 +) 14 

Inner part of a large city (50,000 +) 13 



12 

38 

15 

9 

21 

6 



American Indian 

Mexican-American 

Negro 

Oriental 

Puerto Rican 

IVhite 

Other 



3 

2 

7 

less than 1 

less than 1 

88 

less than 1 



3 

3 

6 

0 

0 

88 

0 



Professionals and managers 
IVhite collar workers 
Skilled workers 
Ihiskilled workers 



23 


20 


29 


27 


31 


31 


17 


22 



Northeast 
Southeast 
North Central 
South Central 
Rocky Mountain 
Far West 



29 


24 


26 


18 


16 


26 


13 


21 


13 


6 


3 


6 



Principals in the national sample who responded to the question- 
naire (see Appendix D for Questionnaire #3) had a favorable reaction to 
the first section of Booklet III (Test Selection Procedure) . Most prin 
cipals rated this section above average on all criteria: 



Criterion 



Ratings 



Percentage 



Clarity 



Very good 
Good 
Average 
Poor 

Very Poor 



44 

38 

18 

0 

0 
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criterion 


Ratings 


Percentage 


Organization 


Very good 


59 


Good 


38 




Average 


3 




Poor 


0 




Very Poor 


0 


y^ropriateness 


Very good 
Good 


56 

41 




Average 


3 




Poor 


0 




Very Poor 


0 


Usefulness 


Very good 


38 




Good 


50 




Average 


12 




Poor 


0 




Very Poor 


0 


Length 


Very Good 


21 


Good 


44 




Average 


33 




Poor 


3 




Very Poor 


0 



There were 11 respondents who felt that the length of the section 
was just average or worse; of these, 8 prefer a shorter section and 3 
prefer a longer one. 

The principals were given an ecerdse to measure their ability to 
pick out the best test in categories ranked high in their schools. Each 
principal listed four categories and chose the best test in each. There 
were a total of 132 responses. In certain categories, there are no tests 
or only one test listed, so there is no problem of choice involved. On 
this exercise, there were 114 responses in which choice was recjuired. 
There was a total of 19 errors by 12 different principals. The overall 
percentage of ri^t responses relative to total responses was 85.6 per- 
cent, and the percentage of ri^t responses relative to responses invol- 
ving choice was 83.3. A perfect score was obtained by 63.6 percent of 
the school principals. 

A further exercise measuring the same skill was givai, and in each 
test category ahnit 88 percent of the principals correctly named the 
best test. There was one item. Recognition of Word Meanings in grade 1, 
that had a higher incidence of errors. The correct answer in that 



A. 



A. 
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category was Peabody Picture Vocabulary Test, which is an individually 
administered test. It may have been deliberate choice rather than error 
which caused 44 percent of tlie principals to choose other tests: tests 
more suitable for group administration. 

The usefulness of Booklet III is limited in schools which have no 
control over the selection of tests. Seven principals (21 percent of 
the sample) indicate that they could not change to a test which they deem 
more appropriate for their schools, because all tests are selected by 
district-level personnel. Tventy-five principals (74 percent of the 
san^le) indicate that they have the freedom to choose (some) tests. 

All of the respondents think that Booklet III would help them in 
presenting a case for a new test to be used in their school districts. 

The principals were asked to state the criteria they wnould use to 
determine the suitability of a test for their schools. 

94 percent named ^feasurement Validity 
65 percent named Examinee Apprcpriaten^s 
65 percent named Administrative Usability 
68 percent named Normed Technical Excellence 

36 perCiMit named all four 
33 percent named three of the four 
24 percent named two 
6 percent named one 
3 percCTt named none 

Hie concept of Examinee ^ropriateness seems to be understood by 
94 percent of the school principals in the national sanqjle. Sixty-five 
percent understand the concept of Normed Technical Excellence, while 
91 percent understand Administrative Usability and 82 percent understand 
Measuranent Validity. 

The reactions of the school principals to ^)pendix A were quite 
favorable: 



Criterion Ratings Percentage 



Clarity Very Good 50 

Good 44 
Average 3 
Poor 3 
Very Poor 0 
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Criterion Ratings Percentage 



Usability Very Good 44 

Good SO 
Average 6 
Poor 0 
V'ery Poor 0 

Organization Very Good 68 

Good 30 
Average 0 
Poor 0 
Very Poor 3 

Time Very Good S9 

Good 27 
Average 12 
Poor 3 
Very Poor 0 



Tile reactions of the principals to Booklet III, overall, were 
quite positive. Hiey were asked if they consider that Booklet III 

Percentage Percentage 





•Yes*’ 


•w 


is useful 


100 


0 


fills a need in their school 


97 


3 


is complicated 


15 


85 


is too long 


6 


94 


is overly sojhisticated 


6 


94 


is what is needed after 


85 


IS 


Booklet II 

is too tixich effort for too 
little return 


6 


94 



The opportunity to make comnents giving overall reactions to Booklet 
III produced 28 favorabJe reactions (82 percent of the sanq)le) , 5 mixed 
reactions (15 percent) , iind one very negative reaction (3 percait) . 

The procedures in Booklet III would be recommended to others by all but 
two of the principals. One of these is the princip>al with the generally 
negative reaction to Booklet III, and the other would not recommend the 
procedures because Booklets II and III showed him that *’the areas of 
greatest need or inportance sean to be intestable.** 
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Califomia Sample - Results of Questionnaire #3 

The California sample continued to have a high rate of attrition, as 
Questionnaire S3 was returned by 39 schools. y\mong these 39 schools was 
a reduced proportion of rural and inner-city schools and of children from 
professional and managerial -class homes. There was an increase in the pro- 
portion of small town and small city schools, of ^(exican-7\Inericans , and of 
skilled and unskilled labor-class ho.ies. 



Rural area 

Residential suburb 

Small town (5,000 or less) 

City of 5,000 to 50,000 
Residential area of a large 
city (50,000 +) 

inner part of a large city (50,000 +) 

iVmerican Indian 

Mexican-American 

Negro 

Oriental 

White 

Professionals and Managers 
White collar workers 
Skilled workers 
Ihiskilled workers ' 



Respondents to 

Original Sample Questionnaire *3 



Percentage 


Percentage 


7 


3 


41 


34 


9 


16 


13 


16 


27 


29 


3 


3 


1 


1 


13 


17 


4 


4 


2 


2 


80 


76 


26 


21 


2S 


27 


29 


31 


17 


20 



Principals fron California schools had generally favorable reactions 
to the Booklet 111 section on Test Selection Procedure. 



Criterion 



Ratings 



Percentage 



Clarity 



Organization 



Very Good 


44 


Good 


38 


Average 


13 


Poor 


5 


Very Poor 


0 


No response 


3 


Very Good 


54 


Good 


36 


Average 


8 


Poor 


0 


Very Poor 


0 


No response 


3 
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Critcrion Ratings Percentage 



Appropriateness Very Good • S4 

Good 38 
Average 5 
Poor 0 
Ver/ Poor 0 
No response 3 

Usefulness Very Good 49 

Good 43 
Average 5 
Poor 0 
Very Poor 0 
No response 3 

Length Very Good 31 

Good 57 
Average 8 
Poor 0 
Very Poor 0 
No response 5 



All of the respondents who objected to the length of the section 
indicated that they felt the section was too long. 

The California sample scored about as well as the national sample in 
ability to choose the best test in categories ranhed high in their schools. 
There were 150 responses, 102 of which involved choice among 2 or more 
tests. The California sample had 20 errors by 14 different principals. 

The correct responses were 87 percent of total responses or 82 percent of 
responses involving choice. Sixty- four percent of the principals obtained 
perfect scores on this exercise. 

On the second exercise of the same type, in each test category about 
90 percent of the principals picked the best test. As with the national 
sample, there were more errors on Recognition of Word Meanings, grade 1. 

The principal has the ability to change to a test he considers more 
appropriate at 79 percent of the schools; in the other 21 percent of 
schools, tests are selected at the district level. In the California 
sample as well as the national sample, 100 percent of the respondents feel 
that Booklet III would help in presenting a case for a new test to be used 
in their school district; several reported having done so. 
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When the principals were asked to state the criteria they would use 
to determine the suitability of a test for their schools, 

92 percent nnmed Measurement Validity 
82 percent named Examinee i^propriateness 
82 percent named Administrative Usability 
68 percent named Noimed Technical Excellence 

53 percent named all four 
23 percent named three of the four 
11 percent named two 
13 percent named one 

TTie concept of Examinee Appropriateness seems to be understood by 
94 percent of the respondents in the California sample. Seventy-nine 
percent understand Nbrmed Technical Excellence, 100 percent understand 
Administrative Usability, and 86 percent understand Measurement 
Validity. 

The reactions of the California school principals to -^endix A were: 



Criterion Rating Percentage 



Clarity Very Good 52 

Good 34 
Average 13 
Poor 3 
Very Poor 0 

Usability Very Good 44 

Good 44 
Average 10 
Poor 3 
Very Poor 0 

Organization Very Good 69 

Good 25 
Average 3 
Poor 3 
Very Poor 0 

Time Very Good S2 

Good 38 
Average 8 
Poor 3 
Very Poor 0 



The overall reaction of the California sample to the aitire Booklet 
III was similar to the national sample's response: it was favorable. 
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The principals were asked if they consider that Booklet III 

Percentage 



is useful 

fills a need in their school 
is cociplicated 
is too long 

is overly sophisticated , , ^ „ 
is idiat is needed after Booklet II 
is too much effort for too little 
return 



”Yes” 

100 

100 

26 

15 

18 

81 

6 



Percentage 

"No” 

0 

0 

74 

85 

82 

19 

94 



The opportunity to respond subjectively to Booklet III produced 84 
percent favorable cements, 16 percent nixed conments, no purely unfavorable 
connents, and one blank. One of the principals suggested "(1) listing 
tests in order of their suitability (2) aitting down on length of expla- 
nation and (3) using effective nickel size words instead of the two-bit 

ones when possible." 

National Sample - Results of Q uestionnaire #4. 

In the national sample, there was no attrition from Questionnaire 3 
to Questionnaire 4. Appendix E esdiibits Questionnaire #4. 

Booklet IV is divided into four sections: Procuring Instruments, 

General Guidelines for Test Administration, Test Sampling Procedures, and 
Using Norms. The school principals responding to the questionnaire felt 
that most of the information in these sections was not new to tiiem, but 
many acknowledged that the old information was packaged in a convenient 
form. 

In regard to the section on Procuring Instnmients , the principals 
indicated how valuable each part of the section was for them: 



Section 
Ordering tests 



Types of tests 



Rating 

Very valuable 
Scmiewhat 
Not at all 
Eliminate 

Very valuable 
Somewhat 
Not at all 
Eliminate 



Percentage 

15 

67 

18 

0 



18 

58 

24 

0 
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Section 

Scoring Services 



Rating 



Percentage 



Very valuable 
Somev^iat 
Not at all 
Eliminate 



18 

62 

18 

0 



Machine Scorable 
Answer Sheets 



Very valuable 
Somewhat 
Not at all 
Eliminate 



26 

65 

9 

0 



TWO respondents noted the absence of comparative cost data, especially 
on comparative costs of scoring services. 

The section on General Guidelines was considered by the principals tc 



Ninety-four percent of the sample said that there were no problem' in 
their testing programs which might have been prevented by use of the infor- 
mation in Booklet IV. Only one principal cited an area of concern which 
was left out of this section: the origin of national norms. 

The predcHiiinant Test Sampling Procedure for schools in the sample is 
student-system assessment (73 percent) . The other schools use a mixture of 
student -system and sampled-system assessment, usually relying more heavily 
on the former (24 percent) than on the latter (3 percent) . Most principals 
(91 percent) are aware of the advantages of sampled-system assessment, and 
most (97 percent) are aware of CSE*s preference for sampled-systan assess- 
ment, but only 79 percent agree with CSE. Those who disagree feel that 
the purpose of testing is not system assessment but individual assessment 
and reporting to all parents. When asked if they intend to implement sam- 
pled-system assessment in their schools, most principals replied with a 
polite ”no”: 



be: 



very laseful 

useful enough to include in booklet 
not useful; include anyway 
useless; leave it out 



3 percent 
69 percent 
22 percent 
6 percent 



yes, by all means 
yes, if possible 
no, by no m?ans 
blank 



3 percent 
71 percent 
23 percent 
3 percent 
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A school or program evaluation has been performed by 62 percent of 
the principals in the sample, and all of those 62 percent used norms as 
part, of the evaluation. TVelve percent used national norms for this eval- 
uation, 18 percent used state and/or local norms, and 9 percent used 
school norms; others did not specify the type of norm used. 

Sixty- two percent of the sample claim to have seen tests with 
tables of school norms; but many cite as examples tests tdiich have no 
school norms: Stanford, Iowa Silent Reading, Metropolitan, California 
Achievement, California Test of Mental Maturity, Otis, and S.R.A. Achieve 
ment Tests. Thirty-three percent of the sanqrle list the Iowa Test of 
Basic Skills, which does have school norms; several of the Kansas schools 
refer to a particular set of school norms for the Stanford Achievement 
Test, devised for their district. Finally, none of the principals 
list the Cooperative SCAT or STEP, which also have school norms. 

The national sample is in unanimous agreement that test publishers 
should be encouraged to provide school norms. 

In regard to this section on using noims, 3 percait of the sample 
felt that it contained too much infoimation, 79 percent felt that it was 
about right, and 18 percent felt that it left out information, specifi- 
cally about how to develop school norms, percentile scores, and the 
averaging of percentile scores. 

Respondents in the national sanple had widely varied opinions 
about national norms: 



The concept of ’’differentiated school norms” is considered 'Very 
inportant” by 65 percent of the sample and ’’somevdiat important” by 
32 percent. The remaining 3 percent had no ccmment. 

Overall, Booklet IV found favor with 68 percent of the respondents 
and drew mixed reactions from 26 percent. There were unfavorable comments 



What are they? 

How valid are the}*^? 

They are better than state and local noims. 
They are worse than state and local norms. 
They are a means of conparison. 

They are not satisfactory. 

They are always more reliable. 



IS percent 
21 percent 
3 percent 
IS percent 
29 percent 
12 percent 
6 percent 



fron 2 respondents (6 percent of the sample) , Those who made conments 
other than laudatory comnents, asked for less explanation of the sort 
of thing covered in test manuals, and a better explanation of the mate 
rial (sampled -system, school norms) not already generally known by 
school principals. 

California Sample - Results of Questionnaire #4 . 

Attrition continued to thin the ranks of the California san^le. 
The 30 respondents to Questionnaire IV included no rural schools and 
a larger proportion of schools in residential areas of large cities. 



Respondents to 

Original Sample Questionnaire H 





Percentage 


Percentage 


Rural area 


7 


0 


Residential suburb 


41 


31 


Small Tavn (5,000 or less) 


9 


13 


City of 5,000 to 50,000 


13 


16 


Residential area of a large 


city (50,000 +) 


27 


38 


Inner part of a large city 


(50,000 +) 


3 


3 


American Indian 


1 


1 


Mexican-American 


13 


16 


Negro 


4 


3 


Ori«ital 


2 


1 


White 


80 


79 


Professionals and Managers 


26 


24 


IVhite collar workers 


28 


27 


Skilled workers 


29 


31 


Unskilled workers 


17 


18 


The respondents indicated how valuable various 


parts of the "Pro 


curing Instruments" section was to them: 




Section 


Rating 


Percentage 


Ordering Tests 


Very valuable 


22 




Sonevdiat 


72 




Not at all 


6 




Eliminate 


0 






i 
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Section 
Types of Tests 



Rating 



Percentage 



Scoring Services 



Mc'ichine Scorable 
Answer Sheets 



Very valuable 


38 


Somewhat 


59 


Not at all 


3 


Eliminate 


0 


Very valuable 


25 


Somewhat 


59 


Not at all 


16 


Eliminate 


0 


Very valuable 


31 


ScMeivhat 


56 


Not at all 


12 


Eliminate 


0 



The section on General Guidelines for Test Administration was 
considered by the respondents to be: 



very useful 

useful enough to include in booklet 
not useful; include anyway 
useless; leave it out 



16 percent 
59 percent 
22 percent 
3 percent 



Thirty- four percent of the sample replied that they could have pre- 
vented problons in their testing programs if they had had the information 
contained in Booklet IV. 

The California school principals describe their testing programs as 
’’always student -system assessment” in 72 percent of schools, and ”more stu- 
dent-syston than sampled-systan assessment” in the other 28 percent. Nine 
percent of the principals intend to inplement sampled-system assessment; 

78 percent will do so, if possible, and 9 percent will not use sampled- 
system at all. Three percent did not reply to this question. 

Eighty-eight percent of the principals are aware of the advantages of 
sampled-system assessment, 97 percent are aware of CSE’s preference for it, 
and 66 percent agree with CSE. Those who feel that a sampled- system 
assessment is not preferable to a student-system assessment cite problems 
of individual diagnosis, small school size, and lack of confidence in the 
reliability of the sampling technique. 

A school or program evaluation has been attempted in 72 percent of 
the schools and has been actually performed in 66 percent of the schools. 
Fifty-six percent of the total sample used test norms as part of this 
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evaluation. ITxese norms were national norms in j 1 percent of the schools 
(in the whole sample), state and local norms in 12 percent of the schools, 
and school norms in 3 percent. 

As in the national sample, the California teachers (56 percent of 
them) claim to have seen school nomis, but cite as examples tests which 
have no school norms: Survey of Primary Reading Develop.ient, California 

Achievement Test, Stanford. Lorge-Thomdike, M.T.A. , Comprehensive Tests 
of Basic Skills, and S.R.A. Ninety-one percent of the principals in the 
California sample feel that test publishers should be encouraged to pro- 
vide school norms; the other 9 percent do not think so. 

Most (88 percent) of the respondents felt that the section on Using 
Norms contained about the right amount of information, 3 percent felt 
there was too much information, and 9 percent felt there was not enough, 
particularly in the areas of stanines, percentile bands, school norms, 
and the interpretation of norms. 

Attitudes of the respondents to national norms was varied. 



What are they? 

They are better than state or local norms. 

They are helpful as an addition to state 
and local norms. 

They are not as valuable as state and 
local norms. 

They are of no consequence. 

They are not useful for program planning. 

They are better because they are based 
on schools like mine. 

They are always more reliable. 

How good is the sampling for them? 

They are useful for comparison purposes. 



31 percent 
9 percent 

9 percent 

3 percent 
6 percent 
9 percent 

3 percent 
6 percent 
6 percent 
13 percent 



Differentiated school norms are considered "very important" by 78 
percent of the California respondents, "somewhat important" by 12 per- 
cent, "not important" by 3 percent, and 6 percent had no comment. 



Favorable reactions to Booklet IV as a whole were recorded by 56 per- 
cent of the sample, mixed reactions were recorded by 31 percent, and dis- 
tinctly unfavorable comments were given by 6 percent, while 6 percent had 



no comment. 



National Sairplc - Results of Questionnaire n5. 



Summarized in the table below are the demographic characteristics of 
the schools responding to the five questionnaires. 



Number of respondents 


Original 

Sample 

79 


Respondents to Questionnaire 
II III § IV V 

46 34 26 

Percentages 


Rural area 


12 


11 


12 


15 


Residential suburb 


42 


39 


38 


33 


Small town (5,000 or less) 


12 


15 


15 


11 


City of (5,000 to 50,000) 


16 


11 


9 


11 


Residential area of a large 


city (50,000 +) 


13 


20 


21 


26 


Inner part of a large city 


(50,000 +) 


3 


4 


6 


4 


Anerican Indian 


3 


4 


3 


0 


Mexican-American 


2 


2 


3 


3 


Negro 


7 


6 


6 


7 


White 


88 


87 


88 


90 


Professionals and Managers 


23 


21 


20 


16 


White collar workers 


29 


29 


27 


28 


Skilled workers 


31 


32 


31 


34 


Unskilled 


17 


17 


22 


22 


Northeastern 


29 


30 


24 


26 


Southeast 


26 


17 


18 


22 


North Central 


16 


22 


26 


15 


South Central 


13 


17 


21 


22 


Rocky Mountain 


13 


9 


6 


8 


Far West 


3 


5 


6 


8 


Questionnaire #5 appears 


in Appendix F. 


The respondents in the nation- 


al sample all indicate that they understand what is 


meant by models and 


decision rules. There is general affirmation 


that the explanation of deci- 


sion models and rules should be kept in the booklet, 


as 96 percent 


of the 



principals found the explanation to be useful or very useful. 



The principals were asked if it had be made clear what the outcome of 
implementing Booklet V would be. Eighteen percent indicated that it was defi- 
nitely clear to them; 71 percent think that it was clear; and 11 percent were 



not sure. 
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'rhc section on extraneous decision rules is considered v'er>' pertinent 
by 41 percent, useful by 78 percent, and irrelevant by only 15 percent. 

This section on extraneous decision niles was useful to understanding of 
the Booklet’s decision rule by 85 percent of the respondents. FJeven per- 
cent suggested that the section was not useful to them, but should be left 
in the Booklet anyway. 

Tlie Booklet's explanations of the variables in the decision rule were 
adequate for all but one of the respondents , but two respondents called 
for a better explanation of the variable "probability of improving perfor- 
mance," and two were slightly confused about the variable "typical level of 
student performance." 

There were no worthwhile additional variables for the decision model 
suggested by the national sample. One principal suggested adding an 
index of school organization and/or continuous progress to the decision 
model; those concenis are important but not relevant to this decision model. 
Another principal suggested "probability of implementing the decision 
model." 

Eighty-nine percent of the respondents felt that the review of proce- 
dures for computing priority values was valuable. Four respondents re- 
ported they had difficulty in following the procedures for computing 
priority values. Two of them specified the section on "probable increase 
in utility" as the center of difficulty. 

Difficulty in using Appendix 1 was reported by three respondents. 

They itemized their difficulties thus: 



Getting a roster of scores 2 

Obtaining a mean raw score 2 

Finding the appropriate table in the test 
manual to convert raw scores to pupil 
percentiles 0 

Converting mean raw score to. a pupil 
percentile score 1 

Using Table 4 to convert the pupil 
percentile to a school percentile 1 



In the national sample, 41 percent of the respondents express 
interest in the derivation of Appendix 1. There is also sentiment for 
explaining the derivation of the numbers in Tables 1,2, and 3: 



11 percent feel explanation is necessary. 

59 percent feel it would be interesting and helpful. 

15 percent feel it would be confusing. 

8 percent feel it is not necessary. 

8 percent have no opinion. 

Only 6 respondents (24 percent of the sample) actually computed priori- 
ty values for any goal areas. Five of them made no errors in recording 
"Probable Increase in Utility" from the appropriate table. Three of them 
made no errors in computing "Priority Value," and three liad no errors in 
ranking the goals. Another principal went through the process of computing 
priority values, but did not provide data to be checked. Qf the total of 
7 principals who actually computed priority values, 5 agreed that the goal 
area selected by the decision rule was indeed the most critical goal, and 
that they could not have determined the importance of this goal without 
the rule. One principal indicated that he could pick the goal without any 
rule, and that the rule picked the wong goal anyway. Another principal 
indicated that he could not pick the most important goal without the deci- 
sion rule, but that the rule caused the wong goal to be chosen "most 
important." There were eight principals who felt that they could deter- 
mine the top-priority goals without going through the Booklet V proce- 
dures. Of these, only one actually went rhrough the computation proce- 
dure. 

Overall reactions to Booklet V were that it 



is useful 


89 percent 


fills a need in my school 


48 percent 


is complicated 


59 percent 


is too long 


15 percent 


is overly sophisticated 


18 percent 


is what is needed to complete a 




needs assessment 


56 percent 


is too much effort for too 




little return 


22 percent 


would be recommended to. other 




principals 


71 percent 



Those principals indicating that they would not recommend Booklet V 
to other principals gave several reasons : 



It requires too much effort for the results . 
Principals would need a workshop first. 
Standardized tests equal standardized children. 



Booklet is too ccmplicated. 

MxperienceU principals don't need the Booklet. 
District has no money for any testing. 



California Sample - Results of Questionnaire ^5 . 

Demographic characteristics of the samples responding to the various 
questionnaires are summarised below. Attrition from the sample may also 
be noted. 



Original Sample Respondents to Questionnaire 

II III IV V 



Number of Respondents 


103 


63 38 

Percentages 


30 


16 


Rural area 


7 


5 


3 


0 


0 


Residential suburb 


41 


40 


34 


31 


37 


Small tom (5,000 or less) 


9 


11 


16 


13 


18 


City of 5,000 to 50,000 


13 


14 


16 


16 


12 


Residential area of a 
large city (50,000 +) 


27 


25 


29 


38 


32 


Inner part of a large city 
(50,000 +) 


3 


5 


3 


3 


0 


American Indian 


1 


1 


1 


1 


1 


Mexican-American 


13 


14 


17 


16 


13 


Negro 


4 


5 


4 


3 


0 


Oriental 


2 


2 


2 


1 


1 


IVhite 


80 


78 


76 


79 


85 


Professionals and managers 


26 - 


25 


21 


24 


20 


White collar workers 


28 


27 


27 


27 


31 


Skilled workers 


29 


29 


31 


31 


33 


Unskilled workers 


17 


19 


20 


18 


15 



All of the respondents in the California sample indicate that they 
know what models and decision rules are. The respondents say that the out- 
come of implementing Booklet V is explained clearly by the Booklet: 44 
percent 5 :ay "most definitely," 50 percent say "I think so" and 6. percent 
say " most definitely not". The explanation and examples of decision rules 
were .very useful to 50 percent, 44 percent feel that they were useful 
enough to be retained in Booklet V, and 6 percent feel they were not very 
useful . 

The section on extraneous decision rules was called very pertinent by 
32 percent, useful by 56 percent, and irrelevant by 12 percent. All but 







Booklet V' 



is useful 

fills a need in their schools 
IS complicated 
is too long 

is overly sophisticated 
is what is needed to complete a needs 
assessment 

is too much effort for too little return 
would be recommended to other principals 



44 percent 
18 percent 
75 percent 



68 percent 
56 percent 
62 percent 
6 percent 
12 percent 



National Sample - Reaction to the Complete KIT 

Part of Questionnaire #5 deals with the complete KIT rather than just 
Booklet V. The respondents to this questionnaire are those 26 principals 
who have completed all 5 booklets. They felt that the entire KIT: 



Compared to other needs - assessment procedures they have heard 
about, the KIT is: 



Most of the principals went through the KIT without assistance, aside 
from the normal and necessary involvement of teachers, parents, and 
pupils in Booklet II. A group of 4 principals worked through the KIT to- 
gether; another worked with other principals; two worked with their staff 
throughout the KIT; one worked with his school psychologist on Booklet II; 
and one worked with his District Director of Curriculum on Booklet II. 



is useful ^ 

fills a need in their schools 
is complicated 
is too long 

is overly sophisticated 
is too much effort for too little return 
would be recommended to other principals 
is better than needs - assessment proce- 
dures they used previously 



92 percent 



85 percent 
41 percent 
22 percent 
11 percent 
11 percent 
8 percent 
82 percent 



much better 
better 



29 percent 
33 percent 
0 percent 
0 percent 
22 percent 
15 percent 



about-.as good 
not as good 

(don't know any other procedure) 
no comment 



■t:- 



Respondents were asked which Ki'oups place most pressure on them to 
institute new programs or relcrms. According to them, district adminis- 
trators exert the greatest pressure, with teacher organizations and the 
I’TA exerting the next greatest pressure. 

Pressure Groups Ranked by Respondents 

Most Pressure 

District Administrators 17 

PTA 2 

Teacher organizations 3 

Student groups 
Professional groups 
Property owners 
Chur ch , bus ines s , women ' s , 
veterans ' groups 

The allocation of funds for next year's programs in the schools is 
determined by several considerations: 

State mandates are of primary importance in some schools 
and are unimportant in others. 

Local Board of Education mandates are among the most im- 
portant influences in most schools. 

Student needs as indicated by standardized tests is not 
an important influence. 

Student needs as observed by teachers is the strongest 
influence in over half the schools, and is a strong 
influence in most. 

Suggestions from parents is the weakest influence of the 
f ive suggested here . 

The principals were asked to imagine that they had $500 to spend on 
any one goal area. They were asked to describe how they would decide how 
to spend the $500. Unfortunately, many of the respondents missed the 
point of the question; they told how they would spend the money and failed 
to describe any decision process. Of those who did describe a decision pro- 
cess, half specified the KIT. Most of the others said they would decide in 
discussion with their staffs. 

Tests are district mandated in 52 percent of the schools in the 
national sample responding to Questionnaire V. Twenty-six percent of the 
schools will choose tests for next year because the tests are appropriate 
and valid for their purposes. The other 22 percent of scdiools did not 




2nd 3rd 



2 2 

9 8 

7 5 

1 2 

1 2 

1 



0 






imlicnte on what basi:> tests will be chosen for the ccsninj^ year. 

/ 

Principals in the respondent schools gather data on teacher attitudes 
in these ways: 



contact teachers individually 
call meeting of teachers 
contact teacher organization representative 
occasion does not arise 
other (small teacher groups, question- 
naire, standing committee) 



74 percent 
63 percent 
8 percent 
0 percent 

18 percent 



Data on community attitudes are gathered by principals through the 



following means: 

PTA 

teachers 

contact with parents 
occasion does not arise 
other (questionnaire, standing committee, 
students, open meeting, coffee hour). 



56 percent 
41 percent 
56 percent 
4 percent 
22 percent 



California Sample - Reaction to the Complete KIT 



The 16 principals in the California Sample who completed all five 
Booklets of the KIT feel that the KIT: 



is useful 

fills a need in their schools 
is complicated 
is too long 

is overly sophisticated 

requires too much effort for too little return 
would be recommended to other principals 
is better than needs -assessment procedures 
previously used 



88 percent 
62 percent 
32 percent 
12 percent 
18 percent 
12 percent 
100 percent 

100 percent 



Relative to other needs assessment procedures they have heard about, 

/ 

the principals consider the KIT to be: 



much better 
better 

about the same 
worse 

(don't know any other procedure) 
no comment 



25 percent 
44 percent 
6 percent 
0 percent 
12 percent 
12 percent 



Principals in the California sample, like those nation-wide , went 
through the KIT largely unaided. One principal cited assistance from 
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the district pupil personnel director, v-hile another listed the School 
Counselor iind Director ot Research as helpers. 

California principals receive pressure to institute reforms from 
the same kinds of sources that influence principals across the country. 
This California sample includes one church-affiliated school; for them, 
pressure from church groups is paramount. The pressure exerted by pro- 
fessional organizations on the California sample appears greater than on 
the national sample; this may be due to non-random sampling, as the prin- 
cipals in the California sample are among the active members of the 
C.E.S.A.A. 

Pressure Groups Ranked by Respondents 

Most pressure 2nd 3rd 



District administrators 10 3 

Teacher organizations 1 52 

Professional organizations 2 32 

PTA 14 6 

Church groups 1 

Women ' s groups 1 

Businessmen, Veterans, Property 1 

ovaiers , Student 



The allocation of funds for next year's programs follows the same 
pattern in the California sample as. in the national sample. 

The California sample did a better job than the national sample 
of describing the decision process they would use in an actual situation; 
that is, more of them described the process and not just the decision. 

Of those who described a decision process, 54 percent would use the KIT 
and 72 percent would use staff meetings; some would use both. 

The majority of the tests used in next year's testing program will 
be selected by 

state mandate 38 percent 

district mandate 19 percent 

school on basis of appropriateness 25 percent 

no comment 18 percent 
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Principals gather data on teacher attitudes through 

individual teacher contacts 
teacher meetings 

contact with teacher organizations 
other (small groups, questionnaires, random 
sampling) 



88 percent 
82 percent 
12 percent 

44 percent 



Data on community attitudes toward the school's instructional program 
are gathered by principals through 

PXA 75 percent 

teachers percent 

contact with parents 62 percent 

other (questionnaire, small group meetings , 
random sampling) 50 percent 



I 
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Tl-m CASE-SIUDY FIELD TESTS 

The results of the large-scale field tests will no doubt provide 
the School Evaluation Program with information concerning the activi- 
ties and considered judgments of principals who have read the materials 
and tried out the activities of the KIT; such impressions will be use- 
ful. However, the Program staff realized that this data will provide 
few insights into tlie dynamics of the process of introducing the KIT in- 
to elementary schools. In an attempt to assess this dynamic dimension, 
the Center contracted to have a series of case studies conducted in 
schools that had agreed to try out the KIT. Originally, three case 
studies were to be conducted; however, because of the complicating cir- 
cumstances described below, only two case studies were completed., 

Design of the Case Studies 

There were several characteristics of elementary schools that were 
chosen for examination in these case studies: 

The Principal's Perceived Leadership . The KIT's designers assigned 
the school principal a dominant role in introducing the KIT into the 
school. Thus it was thought that teachers' opinions of the principal's 
leadership qualities would be an important variable in determining the 
potential acceptance of the KIT in the school. For example, if the 
teachers did not have confidence in the principal's leadership, this 
lack of confidence might be reflected in teacher negativism toward us- 
ing the KIT or toward the KIT itself. 

In an effort to measure the teachers ' perceptions of the princi- 
pal 's leadership behavior, the teachers were asked to complete the Lead- 
er Behavior Description Questionnaire/ - Form XII, LBDQ XII. 

The Informal St ructure of the School . Each elementary school , as 
all organizations, has its own informal social structure. Certain teach 
ers tend to associate with each other in cliques. Some teachers are 
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regarded by their colleagues as opinion leaders; as such, these teach 
ers have considerable influence over events in the school. We hypoth- 
esized that the opinion held toward the KIT by those who were identified 
as influential in the school might have a significant impact on the 
kit's reception in the school. 

The identification of faculty cliques and influential teachers was 
made throiigli an interview with the principal, and by analysis of a 
sociometric device that was completed by tlie teachers. (See Appendix G) 

Pro cedures for Introducing the KIT . Wliile the booklets of the KIT 
describe ways in which they might be introduced into the schools, the 
instructions give the principal considerable leev\^ay in deciding on the 
specific way he will introduce and use tlie KIT. We hypothesized that 
the way in which the KIT was introduced miglit make a difference in the 
faculty's understanding of the KIT’s activities. We sought answers to 
such questions asi Was the principal, enthusiastic in introducing tlie 
KIT? Were his instructions clear? Did he follow the directions out- 
lined in the appropriate booklet? 

An observer was present on each occasion vdien the principal pre 
sented the KIT to the teaching staff, and on some occasions when the 
principal made a presentation to other groups such as parents and 
school board members. Also, teachers and parents were interviev^^ed at 
the end of the school year. 

Planning and Implementation . Each of the schools in the case stud- 
ies had volunteered for the original field testing of the KIT. We were 
unsure as to how these schools learned about the KIT and the principal’s 
motivation for joining in this effort. We hypothesized that the way in 
vvrhich the school becaine involved might prove to be an important factor . 
Did the principal decide independently to join this testing or was it 
"forced upon him" by his superordinates? This might influence his en- 
thusiasm and the amount of time and effort he put into the field test- 
ing. 

Once the principal decided to participate, we were interested in 
learning how he planned to implement the KIT? What problems did he 
foresee? What strategies did he plan to use to counter anticipated 
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obstacles? What was his timetable? Would it be possible to identify 
characteristics common to the organization and functioning of most ele- 
mentary schools that cause problems in implementing the KIT, and are 
there changes that could be made in the KIT booklets which would help 
the user to anticipate and overcome persistent obstacles? 

In addition, would it be possible to identify characteristics 
unique to specific schools that could influence the KIT's introduction. 
Examples of such characteristics are the school population's socio-econ- 
omic and racial status, the size of the staff, the age and experience 
of the teachers, the number of years the principal had been at the 
school, the immediate history of the school. 

These data were gathered from a questionnaire that was completed 
by the principal (see Appendix H) , and from interviews with the princi- 
pal before and after the introduction of the KIT into the school (see 
Appendix I) . 

The Case Studies 

Three schools were selected originally for inclusion in the case 
studies. These schools were selected from a list subnitted by Dr. 
Edward Beaubier of the California Elementary School Administrators 
Association (CESAA) , which is now part of the Association of California 
School Administrators (ACSA) . (CESAA assisted the Center with the 
California sample.) Several factors were considered when selecting the 
schools: socio-economic status of the community, racial composition of 

the students enrolled in the school, proximity to UCLA. A key factor, 
of course, was the principal's willingness to participate in the case 
studies. The three original schools selected were: 

School A . School A is a K-6 elementary school located in a large 
nearby community. The school enrollment is 538. The parent population 
is primarily upper-middle-class professionals (80 percent), with the re- 
mainder of parents being white collar and skilled laborers. Ninety per- 
cent of the students are white. At an initial interview, the principal 
expressed enthusiastic support for the project. Just prior to the be- 
ginning of the case study, however, the principal suffered an accident 
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that caused him to be bedridden for several weeks . Apparently because 
of this mishap, the principal decided after two months that he would 
not be able to participate in the case study. 

School B . School B is a K-6 elementary school located in a commun- 
ity adjacent to Los Angeles. The total student enrollment is 510. The 
community is predominantly white middle-class skilled laborers and white 
collar workers. The school has a small but growing percentage of stu- 
dents with Spanish surnames. (A more detailed analysis of School B ap- 
pears later in this report.) 

School C. The remaining school in the original sample of three 
schools is located in a large metropolitan school district. It is a K-6 
elementary school with an enrollment of 650 students. The parents are 
primarily lower- and middle-class unskilled and skilled workers. Fifty 
percent of the pupils have Spanish surnames and 50 percent are Anglo, 

The principal first heard about the KIT through the CESAA project. 
He was enthusiastic about the KIT, particularly in comparison with some 
of the other CESAA systems analysis and evaluation projects. He was at- 
tracted to what he considered the KIT's practical usefulness. He in- 
tended to use the KIT with teachers, with a community advisory board, 
and with other selected parents and community leaders. 

After three months we had not yet received his questionnaire from 
Booklet I. In a telephone conversation, he said the delay was caused 
by his being too busy with other matters but that he would return the 
booklet soon after Christmas vacation. 

After 4 1/2 months, the principal still had not returned Booklet I. 
In a telephone conversation, the principal indicated that he might have 
to drop out of the project becau.se of administrative difficulties. He 
did not indicate the cause of the trouble, except to say that it had 
nothing to do with the KIT. 

Even though we had not received Booklet I and it was late in the 
year, we were anxious to continue the case study in this school because 
(1) the school differed somewhat from the other schools in terms of 
socio-economic and racial characteristics 5 (2) there appeared to be a 
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major difference of opinion among the school staff regarding the goals 
of the educational program. We wanted to see if the different views on 
educational goals were reflected in the card-sorting. We were interest- 
ed in knowing if the KIT could help the principal and teachers identify 
and resolve their differences of opinion. 

We continued to hope that the case study would proceed in this 
school. However, after six more weeks we were notified by the prin- 
cipal that the school would be unable to participate in the study. 

Thus, very late in the year, we lost one of the initial case-study 
schools. 

School D . With the loss of School A, we sought to conduct a case 
study in School D, a multi-racial school in a nearby district that had 
undergone court -ordered racial integration. Because of the potential 
goal- conflicts that one might anticipate in schools that are going 
through substantial change in a short period of time, we were anxious to 
conduct a case study in this school. It would have been interesting to 
see if the KIT were useful in resolving the goal -conflicts . 



We were informed in early December tliat the school did not wish to 
participate in the proposed case study. 

School E . In early winter, 1971, School E agreed to participate in 
the case study. School E is a K-8 school with a pupil enrollment of 881. 
Seventy- five percent of its parents are skilled laborers and white col- 
lar workers. Approximately 66 percent of the parents are Aiiglo, almost 
all the remaining parents have Spanish surnames. The school is located 
in a community that adjoins Los Angeles. (A more detailed description 
of School E is included in the next section of this report.) 

In summary, only one of the original three schools was included in 
these case studies. School E was added at a later time. Thus, the re- 
maining report is centered on case studies conducted in Schools B and E. 

School B 

School B is a K-6 elementary school with a student enrollment of 
510 and a certified staff of 18 full-time teachers and one half-time 
EMR teacher. The school is part of a K-12 unified school district with 
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an enrollment of 15,183.. The school services an. attendance area that 
is characterized as predominately white middle-class. The majority of 
parents are white collar supervisors, managers, and hourly workers. 

There has been little pupil turnover, and the size of the school has 
remained constant during the past three years. The school's educational 
program, as described by the principal, is traditional. 

The pupils are typical for such a community. Data on intelligence 
tests indicate a normal distribution. There does not appear to be an 
unusually large number of retarded, or disruptive, or brilliant pupils. 

All but one of the teachers are women. Tv^/o- thirds of the teachers 
have taught for five years or more. One teacher does not have an A.B. 
degree. The principal holds the only advanced degree. Four of the 
teachers live in the district. There is very little teacher involvement 
with the community. The principal reports a division in the faculty be- 
tween older and younger teachers (8 teachers under 30, 8 over thirty). 

The older teachers have tended to be very influential on the school in the 
past. Because of large turnover during the past few years (retirement), 
there is a shift toward a younger teaching staff. 

A sdciometric device was administered to determine if any decided 
cliques or influential teachers could be identified. (This device is 
reproduced in Appendix G.) A decided difference was noted among pri- 
mary (K-2) and upper elementary (3-6) levels. That is, the primary 

■ teachers tended to interact with one another, and the upper elementary 

( 

i teachers interacted with one another. Five teachers were identified, 

j through use of the sociometric device, as being influential and three 

'• teachers were identified by the principal as influential. 

The principal is in his second year as princ.ipal of this school. 

He came to the school from previous experience as a principal in another 
I school in the district. As is often the case, the new principal had to 

i spend a considerable amount of time in adjusting the faculty and staff 

j to his style of leadership, which was somewhat different from that of 

f his predecessor. He encountered a number of difficulties in doing this 

! but by the end of the first year he felt that the faculty and staff had 

( grown accustomed to his- leadership. 
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The faculty card- sort took place in March. The principal had dupli- 
cated the card deck so that each teacher had his own deck. The principal 
explained the KIT briefly and gave oral instructions as to how the cards 
were to be sorted. Several general questions were asked. The most cri- 
tical one centered on what reference the teachers were to use when sort- 
ing the cards, i.e. , in terms o£ a particular grade or for the entire 
school. The principal answered, "You are to think of the school, its 
whole program for K-6." 

The teachers sorted for about 40 minutes (some going to other rooms, 
some staying in meeting room) . They all returned to the meeting room 
where the principal explained the tally procedure. The card- sorters 
serialized tl\e cards in each pile from 1-A through whatever number in 
the stack. When they were gathered together, one person read off column 
1, 1-A to 41-C, Each person with a card in column 1 would raise his 
hand when the appropriate number was called off. The hands were counted 
and recorded. This was done for each of the 5 columns. The tally was 
completed in about one hour. The entire card-sorting, including the 
tallying took two hours. A summary of the teachers' total tally was 
prepared and sent to all members of the faculty. 

In early April, eight parents on the PTA board of directors com- 
pleted the card-sort at a PTA board meeting. These parents were not 
randomly selected. 

Teacher Interviews .. In May, interviews were held with eight 
teachers who participated in the card-sort. (See Appendix J) The 
following is a summaiy of their answers to questions they were asked 
during the interview: 

1. Seven of the eight teachers initially learned of the KIT 
at the January faculty meeting. The teacher who had been asked to 
assist the principal with the implementation of the KIT had heard 
of the KIT prior to the faculty meeting. 

2. There was considerable difference of opinion about the 
purpose of the card-sort. Two teachers had "no idea" of the objec- 
tives of the card-sort; two thought it was the first step in "writing 
a new curriculum"; two thought the card- sort was to identify defer- 
ences between teachers and parents ; one thought the KI*T was fo iden- 
tify the teachers ' goals ; and one thought it was to identify the 
most and least important "behavioral objectives." 
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3. Five of the teachers were satisfied with the way the KIT 
was introduced. Two felt they were somewhat unprepared to adequately 
complete the card- sort, and they would have appreciated having an 
opportunity to become acquainted with the cards before they began 
the sorting; one experienced teacher recommended that the cards 
sorted by experienced teachers should be given more attention than 
those of new teachers. 

4. Two teachers had no further questions about the KIT (largely 
from lack of interest) ; two teachers have no idea of what is to 
happen next; two teachers thought this was a UCLA research project 
and that we were collecting data for UCLA; and two were confused as 
to whether they should have sorted on a grade or school-wide basis. 

5. All of them had received the tally sheet of the total faculty 
card-sort. Three wondered how the parent Q-Sort had turned out (this 
was not tallied and distributed to the teachers). 

6. None of the teachers was aware of any plans for using the 
results of the card-sort in their school. Participation in the pro- 
ject had not really had much of an effect on the school. There had 
been no resulting formal or informal discussion of the KIT. 

7. Several teachers expressed concern, over their ability to 
sort the cards on a school -wide basis. They felt they were "locked 
in" to their grade level, and they tended to respond on a grade 
rather than school-wide basis, even though they tried to be more 
general in their responses. 

8. Tvro teachers questioned the appropriateness of .some items 
for elementary schools. One suggested there should be a greater 
number of items on drug education. 

Principal Interview . In general , the principal was pleased with 
progress thus far in using the KIT in the school. He plans to appoint 
a faculty steering committee to help him with the next steps. The pos- 
sible faculty divisions that he mentioned in an earlier interview did 
not materialize, although he suspects that they might have developed 
had they attempted the card-sort during the preceding year. (The other 
district school that participated in the card-sort experienced a very 
similar reaction from the faculty.) 

The principal was disappointed with the parent participation. He 
felt it was difficult to find parents who had the time to complete the 
card sort. 

A district-wide PPBS Needs Assessment Steering Committee has been 
formed to modify the goals in Booklet II of the KIT for use throughiout 



the district. The committee had the following reservations about the 
KIT: 



1. The cards are hard to understand. Some of the items are too 
long and complex and the vocabulary is confusing, especially for lay 
people . 

2. There are too many items and the resulting tediousness limit.s 
the effectiveness of the KIT. 

3. TTie card sort and resulting tallying is unnecessarily compli- 
cated. They prefer a straight questionnaire. 

As a result, the district will be revising the cards into a queis- 
tionnaire that will be distributed in an attempt to determine district 
goals. 

Parent Interviews . Four of the eight parents who completed the 
card- sort were interviewed. Their comments can be summarized as 
follows : 



1. The instructions for the KIT were clear; however, they found 
it difficult to sort the cards on a school -wide basis, even though it 
was their understanding that that was what they were to do. Instead, 
they tended to sort in terms of their particular child, the grade he 
was in, and his unique characteristics. Those parents with more than 
one child in the school had considerable difficulty in sorting some 
cards . 

2. Although they found the exercise to be interesting, they did 
not understand some of tlie terms on the cards. 

3. They felt unaware of how the KIT was to be used in the future. 
They felt a general questionnaire sent to the homes might afford a 
greater number of parents an opportunity to express their ideas about 
the goals of the schools. 

School E 



School E is a K-8 elementary school vdth a student enrollment of 
approximately 900, and a certified staff of 29 full-time teachers. The 
school is part of aji elementary school district with a total enrollment 
of 9,039 pupils. 

The school's attendance area is in a state of transition, Origin- 
ally rather affluent, the area now has considerable diversity in terms 
of socio-economic level and ethnic background. The principal estimates 
that about 50 percent are white collar, about 35 percent a:.*e skilled 
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workers, and 15 percent are unskilled laborers. There is a decided 
shift recently toward an increase in the percentage of unskilled 
workers. About two-thirds of the parents are Anglo, about one-third 
have Spanish surnames. Tliere are few Blacks, Orientals, and American 
Indians in the community. The percentage of Spanish surname families 
is rising steadily. The principal reports that there is no visible 
parent unrest regarding the school. 

There is a continuing increase in the percentage of student turn- 
over with the current rate standing at almost 25 percent each year. 

The size of the pupil population has remained quite constant over the 
past three years. 

In terms of academic ability, the student body has essentially a 
normal distribution; however, this distribution appears to be changing. 
The principal reports that the school's mean I,Q, score has been drop- 
ping about 2 points per year during the past three years. The principal 
attributes this change to the difficulty children with Spanish surnames 
have on intelligence tests that are written in English. Consequently, 
as the percentage of Spanish surname pupils increases, the average I.Q. 
score tends to decline. 

Until funds ran out in 197U-71 academic year, the school was an 
ESEA target school. The local Headstart program was located in the 
school building. 

Of the 29 certified teachers, 10 are men, 19 are women. One-third 
of the teachers are over 50 years of age, one-third are in the 31-50 
year age bracket, with the remaining third in the 21-30 year age group. 
All of the teachers have a minimum of an A,B. degree, approximately 
one-half have earned the Master's degree, and one teacher has earned a 
doctorate. The principal is in his sixth year as principal of the 
school. 

A sociometric device was administered to assess the informal struc- 
ture of the school. Only one-half of the instruments were returned; 
therefore, it is difficult ^o develop any comprehensive picture of the 
school's informal system. From the sociogram, it appears that the upper 
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elementary and 7th and 8th grade teachers form a loose-knit clique. 

This is consistent with the principal's observations. Five teachers 
were identified through the sociometric device as being influential; 
the principal identified only one of them as being influential. 

In summary. School E is located in a transitional residential 
area. There does not appear to be any serious effort on the part of 
the school staff to assess the implications the changed pupil popula- 
tion might have for the educational program. There does not seem to 
be any serious split in the faculty except perhaps that the 7th and 8th 
grade teachers are considered more liberal in educational matters. 
Several influential teachers were identified. The principal's leader- 
ship seems managerial rather than curriculum oriented. 

Implementation of the KIT . The principal was asked by the asso- 
ciate superintendent for curriculum to participate in the CSE evalua- 
tion project. It was not until early January, that the principal first 
heard of our interest in conducting a case study. Thus he started some- 
what behind the other case-study school. 

In an interview in late January, the principal's plans were as 
follows: In February, the school board, all K-6 teachers (optional 

for 7th-8th grade teachers), 30-40 parents selected by the principal 
on a stratified basis (socio-economic) , and approximately 50 pupils 
were to sort the cards. The- card- sort was to be completed by the end 
of February. The entire school board would complete the card-sort, 
the teachers would be split into two groups, and several small groups 
of parents and pupils would complete the card-sort. An administrative 
aide was to tabulate the teachers' cards; a parent group was to tabu- 
late the parent and pupil cards. He was going to introduce the KIT 
to the teachers at a separate meeting before they completed the card- 
sort. He did not elaborate on how he would introduce the KIT to the 
school board, parents, and pupils. He indicated 1:hat the cards would 
have to be translated into Spanish in order for some parents to com- 
plete the card sort. He planned to complete the entire KIT project 
by the middle of June. 



In early February, the principal presented the KIT to the school 
board. He reviewed the CSE project and its relationship to CESAA. The 
observers , who had talked with the principal prior to the board meeting 
and who were familiar with the KIT, could follow his presentation. How- 
ever, it appeared that not everyone on the board was entirely clear as 
to the kit's purpose and design. The board members were each given a 
set o£ cards and they were asked to complete the card- sort sometime 
during the following two weeks. Several procedural questions were asked 
by board members. Two board members expressed concern that the goals 
might not reflect the needs of some minority students, e.g. , English as 
a second language. The board members were instructed to use the blank 
yellow cards to indicate such items. At the close of this case study 
(five months later) no tabulation has been made of the board members' 
card- sort . 

In late February, the principal met with 19 teachers and 2 parents 
for the purpose of introducing the KIT to them. He explained CSE and 
the KIT, and the school's involvement in the field testing. He was 
asked on several occasions if the teachers were to sort on a school or 
grade level. He answered that their uppermost concern should be what 
is best for "the kids," and what they think a 6th grader should be like. 
Their card- sort should reflect those concerns. His presentation was not 
entirely clear. Several teachers picked up a packet of materials to be- 
gin their sorting, l^on completing the sorting, the teachers were to 
return the packet to the central office where it was to be distributed 
to another teacher. 

Only one teacher completed the card-sort at that time. In June, 10 
teachers were asked to complete the card-sort that evening and to be avail- 
able for an interview with "someone from UCLA" on the next day. Those 
selected teachers completed the sort as requested, returned the tally 
sheets , and appeared for the aforementioned interviews . No tabulation 
has been made of the card- sorts nor has the principal articulated his 
plan for having the other teachers complete the sort. The principal says 
he intends to have the other teachers complete the sort "as soon as 
possible." 



Similarly, five parents were phoned during the first week in June, 
1971, and were asked to complete the card-sort and be available for 
some UCLA interviews. The parents selected are on the PTA Board. Three 
of the members completed the sort; one other member had her husband com- 
plete the sort. No tabulation had been completed of the parents' sort. 
The principal still plans to have "many parents" complete the sort some- 
time this summer. However, he was not specific as to when and how this 
was to be done. 

Teacher Interviews . During the second week of June, nine teachers 
who participated in the card- sorting were interviewed. The following is 
a summary of their answers to questions about the KIT. 

1. Six teachers first heard of the KIT at the faculty meeting 
held for that purpose; one teacher was sure she had completed this 
card-sort sometime the year before- -but she could not elaborate; two 
teachers who missed that meeting heard about it from fellow teachers 
the next day during a coffee break; and one teacher first heard about 
the KIT when she was asked to do the card- sort. 

2. The seven teachers who had attended the principal's presenta- 
tion thought the purpose of the KIT was to identify the goals of the 
school as they relate to curriculum development. The two who missed 
the principal’s explanation had no idea as to why they completed the 
card- sort. 

3. One teacher thought the principal's introduction was suffi- 
ciently clear. The remaining five teachers were unsure of the KIT's 
purpose. All of the teachers were unclear about how long to take in 
completing the card-sort. Several felt there had been too long a time 
delay between the introduction and the actual card- sort. 

4. Three teachers expressed concern over what will happen next. 
Five- teachers were unsure about whether they were to consider goals 
that were appropriate for the average sixth grader, or for children 
at the grade level at which they teach. They felt that it was dif- 
ficult to sort for some hypothetical average child. This point was 
mentioned repeatedly during the interviews. 
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5. None of the teachers had received any feedback of the re- 
sults, although several expressed an interest in such information. 

6. None of the teachers knew of any plans for using the results 
of the card-sort in the schools. 

7. A number of teachers commented on the inappropriateness of the 
wording of Category 5, They could not find items that were "unimportant 
or irrelevant." Several teachers found the card-sorting forced them to 
think through some of their assumptions about schooling and felt that 
the card-sorting activity was useful. One stated he would be interested 
in completing the card-sort again when he had more time. They all seri- 
ously questioned whether the KIT will have any impact on their school 
because of lack of clarity as to the purpose, and the unsystematic 
scheduling ;and introduction of the card- sort. 

Principal Interview . As of the second week of June, the principal 
still plans to have all the teachers complete the card-sort. He did not 
elaborate how this would be done by the end of the school year. Five of 
SO parents have conpleted the card-sort, he plans to have the remainder 
comrjlete the sort this summer. Four of the school board members have 
coinpleted the card- sort, one still has not returned the cards. Although 
this sort was scheduled in early February, no summary has been completed 
of the results of their sort. The principal intends to have the students 
conplete the sort this summer, although none had done so as of June. He 
plans to complete the implementation of the KIT during the next year. 

Based on discussions with faculty members, the principal feels some 
of the items on the cards are toe long and confusing. In conversations 
with the principal, the faculty expressed considerable skepticism regard- 
ing the usefulness of the KIT. The limited communications system in the 
school, occasioned by the grade organization, seriously limits the poten- 
tial for using the KIT in the school. 

Part of the reason for the considerable delay in completing the card- 
sort, according to the principal, is the limited number of card sets he 
was given. Had he been given at least 15 sets, he feels he could have 
conpleted the card sort much sooner. 
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It will be necessary for him to translate the cards into Spanish in 
order for some parents to complete the card sort. He feels that some of 
the cards v;ill be difficult for some parents; he suggests we convene a 
group of parents to help rewrite the cards into a more acceptable form. 

P arent Interviews . Four of the 5 parents who had completed the 
card- sort were interviewed. Two of the parents had been present when 
the principal made his presentation to the faculty; two of them first 
heard about the card- sort when they were handed the deck of cards. All 
of them saw the purpose of the KIT as that of helping the school and 
community assess its goals and curriculum. All of them found the instruc- 
tions reasonably clear; three parents felt the vocabularly in some cards 
was difficult. All of them seemed to think the card-sort is an inter- 
esting idea; however, they were unaware of how or if they would be in- 
formed of the results of their sort and the teachers' sort. They knew 
of no plans to implement any change in the school as a result of the 
card- sort. 

All of the parents felt that they had learned something as a result 
of the card- sorting. Some were surprised at the amount of value they 
gave the affective items. They found it difficult to sort for "an aver- 
age sixth grader." They tended to think of their children when sorting 
and responded accordingly. 

Results _ . 

The first part of this section is organized around the various 
dimensions of the schools that were examined. Schools B and E are 
compared and, in as much as it is possible, conclusions are reached 
from such con^arisons. 

The second part of the summary lists some general conclusions 
about the use of the KIT in the schools. (There is a great risk, of 
course, in trying to reach any broad conclusions from so limited a 
sample; nonetheless, such conclusions, coupled with the larger data 
collection effort , may provide useful insights for refining the KIT.) 

The Principals Perceived Leadership . The LEDQ XII (Leader Behavior 
Description Questionnaire) was administered to 16 of the 18 teachers in 
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School B and 16 o£ the 29 teachers in School E. The low rate of re- 
turned instruments in School E (55 percent) severely limits any statis- 
tical analysis o£ the data. A two-tailed t-test was made of the mean 
leadership score of the two principals in the two schools and the re- 
sulting differences were matched with our observations about the dif- 
ferences in their leadership styles. 

The teachers perception of the leadership behavior of the princi- 
pal of School B (Principal B) was found to differ significantly from 
that of the principal of School E (Principal E) on Dimension 5 (Initia- 
tion of Structure) , Principal B scored higher than Principal E on 11 
of the 12 leadership dimensions. These data tended to confirm our ob- 
servations. 

Based on the LBDQ XII, and our observations, it appears that the 
principal's leadership is indeed a very important variable in deter- 
mining the success of the KIT's implementation in the school. 

Informal Structure of the School c The scciometric devices indi- 
cated that there were identifiable and influential teacher cliques in 
the schools. In these two studies, however, it appears that these 
cliques and influential teachers had little or no impact upon the im- 
plementation of the KITr The reason for this is that the KIT was nou 
of sufficient interest that it was a topic of teacher discussion. Thus 
the informal structure was not called into play. It is entirely possi- 
ble, however, that as schools move further into the implementation phase, 
i.e., when controversial curricular decisions are made, that the infor- 
mal structure of a school may well come to the foreground. It was evi- 
dent in our inteiviews that younger and older teachers tended to view 
each other with some suspicion. As their differences become manifest, 
the informal system could become very important in influencing the 
implementation of the KIT. 

It was apparent in School B that there is an historical dimension 
to be considered when examining the implementation of the KIT. If the 
pTi Tir. 1 pal of School B had attempted to introduce the KIT last year, it 
is reasonable to speculate that he would have had a great deal of dif- 
ficulty. For a variety of complex reasons, the infoimal system may 
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have reacted vigorously to oppose the principal's efforts to introduce 
the KIT. (Likewise, School C might well have rejected any attempt to 
introduce the KIT at this time.) It appears that there has to be a 
threshold of cooperation and tranquility in a school and community 
before the KIT will have much of a chance of being implemented. 

Implementation of the KIT . Both principals had similar plans for 
implementing the KIT. Principal B made some progress (completed Book- 
lets II and III); Principal E made little headway (no Booklets completed). 

Principal B's presentation of the KIT to the faculty was far 
clearer than that of Principal E, and this difference showed up in 
the interviews with the teachers, that is, the teachers in School B 
were more knowledgeable about the KIT's purposes and design. It 
appears, however, that the time lapse between the meeting in which 
Booklet II of the KIT was introduced and the actual card- sort (a 
little over a month in School B; approximately 3.5 months in School 
E) was excessive and tended to minimize the teachers' knowledge about 
and enthusiasm for the card- sore. 

The procedures for the actual card- sort in School B were far su- 
perior to those used in School E, and one suspects uhis made a consider 
able difference in the effort the teachers put into the card-sort. Prin 
cipal B completed the card- sort in such a way that it was looked on by 
the teachers as a positive experience. The teachers had enough time to 
complete the sort and the means he used to tabulate the cards stimulated 
the teachers interest in the project. In addition, he was able to give 
the teachers some feedback within a reasonable period of time. The 
teachers in School 'E tended to view the card-sort with some annoyance. 

In several instances, it was because they felt frustrated by having to 
complete the sort during the last week of school and with only a faint 
idea as to why they were being asked to do it. It is likely that the 
sorting of some or all of these teachers was very perfunctory. 

Neither principal was very successful in securing any kind of 
comprehensive parent involvement in the project. It is still not 
clear as to why this is the case. Either the principals had little 
contact with parents or they were reluctant to ask parents to take 



time to complete the sorting, or they simply did not know how to 
proceed with this task. The parents seemed interested in the card- 
sort and they did not express the opinion that the sorting was a 
heavy imposition on their time. Several parents stated that the 
items were too long and the language was a little too difficult. 
Neither of the principals had tabulated the results of the parent 
card- sorts. 

General Comments . At this early stage, the KIT has had very 
little impact on either school. The teachers and parents were cooper- 
ative and moderately interested in the project but none of them had 
really done any serious thinking about the KIT. None of them viewed 
the card-sort as the first in a series of events that will help the 
school more precisely define and attain educational goals. Their 
lack of knowledge about the KIT coupled with the decision to take 
two years to complete the project limits considerably the likisli- 
hood that the KIT will have any lasting impact upon these schools. 
This isn't to say that the potential interest isn't there. Several 
parents and teachers commented that the Q-Sort helped them clarify 
their thinking about educational goals. This potential interest 
just wasn't utilized. 

It appears that these principals were not very effective dissem- 
inators of the KIT. One was quite skilled at performing the Q-Sort 
efficiently, the other was not so skilled. Neither was sufficiently 
enthusiastic to excite the faculty and parents about the project. 

This may have been a result of their lack of understanding of the 
design and purpose of the KIT. 

Neither principal made any serious attempt to use the Q-Sort to 
assess community interest. The parents selected were either school 
board members or members of the PTA executive board. While the opin- 
ions of these citizens are important, they do not begin to represent 
the opinions of any community of diverse socio-economic or racial 
composition. 
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APPENDIX A 

MAIL TO: 

Dr. Ralph Hoepfner, Director 

School Evaluation Project 

Center for the Study of Evaluation, UCLA 

Room 145, Moore Hall 

405 Hilgard Avenue 

Los Angeles, California 90024 




AND 



Formal Application for Inclusion in Field Implementation Sample 
. Elementary School Evaluation Project 
Center for the Study of Evaluation, UCLA 



School name* 
School address 



ZIP . 



Name of school district 



School Descriptive Information 



1 . 

2 . 



Grade span of school 



Approximate pupil enrollment 
(Sept. 1970) 



3. Which one of the following categories 



best describes the neighborhood 
served by your school? 



a. 



_b. 

c. 



d. 



e. 

"f . 



_g 



rural area 
residential suburb 
industrial suburb 
small town (5,000 or less) 
city of 5,000 to 50,000 
residential area of a' large 
city (50,000+) 
inner part of a large city 
(50,000+) 



Racial-ethnic characteristics 
of student body (approx, 
percentages) . 

(a) American Indian. 

(b) Mexican-American 

(c) Negro 

(d> Oriental . . . 

(e) Puerto Rican . 

(f) White. .... 

(g) Other (specify) 

Total 100% 



About what % of the pupils served 
by the school fall into each of 
the categories listed in the chart 
below: 



% 



a . 



b. 



d. 



children of professionals and 
managers (doctors, lawyers, 
engineers, executives, etc.) 




children of white collar work- 
ers other than those in (a) 
above (proprietors, salesmen, 
clerks, etc.) 




children of skilled workers 
(electricians, carpenters, 
repair men, factory workers, 
etc.) 




children of unskilled (labor- 
ers, janitors, dishwashers, 
etc.) 




TOTAL 


100% 



*Note: 



If applying for district participation, please include school description for 

all schools to be included. ^ ; — ; — 

bn the reverse side of this sheet please include a statement relative to why inclusion 
of this school in the field implementation is being sought. Include any characteristics 
of the school and/or community that might support the application. 



O 

me 



Name and Title of individual submitting application 



M. 



APPENDIX B 

ELEMENTARY SCHOOL EVALUATION KIT 
Field Implementation QUESTIONNAIRE //I 

To he Qompleted by adhoob prbnoipal 

This questionnaire must be completed and returned to CSE 
before the next Booklet In the KIT can be sent. 




The questions below are being asked In order to learn more about the 
environment In which the Elementary School Evaluation KIT Is being 
Implemented. No Information will be released by the School Evaluation 
Project In any form that would allow Identification of any particular 
school. 

1. Please check the appropriate choices below relative to planning 
and declslon-maklng procedures In your school: 



• 


planning Is done 
at school level; 
decision to Imple- 
ment Is made at 
school level. 


planning Is done V 
at school level; 
decision to imple- 
ment is made at 
district level. 


plans and 
decisions 
are made 
at district 
level. 


Instructional 

program 








Budget 








Teacher 

assignment 






i • 



2. In those areas above where planning and/or declslon-maklng occur 
at the school level, who Is involved In the process? (check all 
that apply) ^ ^ ^ ^ ^ ^ 



Planning , Decision-making 





Prin. . 


teachers : 


parents. 


.Prin. 

. ii* . ' 


teachers 


parents 


Instructional 

program 




■: ■'iE'.' '■ 




V- 






Budget 










’v.' . >1 \ ■ ' . ■ , ■■ ■ ■ 




" 'Teacher 
assignment 




V..- 

SaSSI 


K>r';e. ■ 









What;;i8"ihe iitie;-bf;''y^r::'ii^ | 

Er|c-. j 

. . ■ ■ ■ ■ 'A'. ■ 



4. Using the numbers 1, 2, & 3, Indicate on the following list 
the top three groups that place the most pressure upon you 
to institute educational reforms and/or new programs: 

(1 “ most pressure) 

Chui'ch or religious groups _■ 

Business organizations . ■ 

Women's groups 

Other professional groups 

Veteran's organizations _____ 

Property owners' association ■ ^ 

Teacher organizations ■ ■ ■ 

Student groups _____ 

District administrators 

Parent-Teacher Associations 



5. How many families of your students are represented at a typical 
meeting of the PTA or similar parent groups? (check one) 

We have no parents' organization 

Only a few 

^ - ■ Less than half 

About half 

Over half 
Almost all of them 






O ' " 

ERIC 
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When you are interested in learning of the attitudes of your 
teachers toward a particular issue regarding the instructional 
progr^ (change in schedule, new materials, etc.) , what i^ 
normal, procedure? (check all that apply) ' 



C \ ' 



contact teachers individually 
call Meeting of tea.chers ;^' 
contact teacher organ! zati bn (s ) represeht'atiye (s ) 



■ Oc c as ion does -not r j^ i^ wher e thi s i s nec es saury i 



other 



When yoii want? to learn| of your school., cfi^^ toward a 

part iculbr . e r ^ : y our |s c t i bna 1 ■ program i what 

is ypur :'yusualjfprbcedur 




/; >-pccaBipff::;d6e;s|;hbt:S^ 

othe^'C^lbase'^sp^ 






vvv:;;y : 



■^"'r 



8. Please list below any experimental and/or Innovative projects which 
have been implemented in your school during the past two years. For 
each project, briefly indicate where the idea for the project originated 
Ctc^cher suggestion, school curriculum committee, district office, etc.) 
and the reason the project was initiated (low achievement scores in 

math, too many accidents on playground, just wanted to try out new 
program, etc.) 



Experimental or 
Innovative Projects 


Where idea 
originated 


Reason project 
was initiated 














■ ■ 












■ C " '■ ■ ■ ■■ ■; ■ ' ■' - '■ 




• . ’ ' /^ ' ' ' 



// 

9. Please rank in order of toportance the following considerations 
relative to deciding on the allocation funds for next year's 
programs in your school. (1 “ most Important) 



State mandates 

Local Board :bf Educat ion mandates ■ 

student needs as indicated by; standardized tests 

student needs as observed by teachers, r. 

, suggestions from parents ' 

10. You are suddenly giyen $500, to spend on any one i^ 

^ in your school . Please describe briefly how you would decide where 
;t(5i- spend the' money'^ ' • ' i 



; - ' a 



o 

ERIC 



• f; ■ 

ii 

'll 



The following questions pertain to your standardized toting 
program, Infoirmatlon Is requested for grades l^ 3, 5, and 6, 
If your school does not contain one or wore of the grades , 
place a large oyer that box. 



First grade 



11. Please list by iiame the standardized tOsts adnjlnlstered last year to 
your first graders. For each test ^indicate the area of the 
instructional program being measured. 



Title of test | area being measured 


1 ■ ■ ' ' ' ■ ' r 

^ It '■• . .. .1', // ■ 




■■ : ;l \ - . ■ ■ ■ ■, Ii ' 

' ' ii ■■ .■'■■■■, V ■ ■ Ii ' . 






.. 7,'.; ■ 


• .7 7 .'til 




;p . .. ... . : '■ ■ - ' 





I 



12. How were the majority of the above-named first-grade tests selected? 

. ..(check, one) 

a - ‘ ,y, . ■ ' ■";! ' ' 



istate mandated 
/kis tilct iiaandat ed 



price;' 



what other schools h^e doing 
(| whatiithe school has always done 



i the t es t measTir e ^ ed:ea we want measured 



'..-0 



-f 



■ n 



- . f 

: r: 



'■ ; 1 ! 
'■•■.ii 

... ■ .il - . 



, ,;ir, , , . ^ , ... , . 

13. The 8c|res from these first-gradie; tests were actually u 
.(check||all’ that. app^^ 






staff :hiu:ric\iliu^ipiiMinihg'^^ 

, studentwcoianseli^fahl^ -v^ 

■■ pupi]:v^<^ping^;yp|:-;:; 

teacher' ,<^al^^ 
hoinp^'dhbns/.ajiqngr^ 
pupii;i grading 






no" 









Third grade 

14, Please list by name the standardized tests administered last year 
to your third graders. For each test, Indicate the area of the 

instructional program being measured, 

; Instructional 

Title of test area being measured 



15, How were the Tnajority of the above-named third-grade tests selected? 
(check one) 

state mandated 

district mandated ^ 

price 

what other schools are doing 
what the school has always done 

the test measures an area we w^t measured ' 



16 . The scores from these third grade tests we|ra^ac^^^ 
'■ (check; all ^>hat''apply)?;;\; v 



ERIC 



ataf f ;curricuitun-pl^ning;;;se;?l3ions; 
student " counseling ;and.v;-dij^ 

,pupli;'grouping;;.:\;v;^ : Vl;;:;,' 

teacher' 'evaluation;;;;; ,■ 

: c canpar isons';: among ■ : schools j . ,■ .-'i. ■; :■ ^ 

' pt^ii; grading ;;; /; ' ■•'33.v • 















• • M-, ■ 



C{ 






Fifth grade 

17. Please list by name the standardized tests administered last y<iar 
to your fifth graders. For each test. Indicate the area of the 
Instructional program being measured. 

Instructional 

Title of test area being measured 



18, How were the ^ of the abbye-hamed f if t^ tests selected? 

. (fheck;: one) , : . 'V, \ v 

i. state mandated \ : 

'district'.- mandated .' - - ■ 

, ■ ■ ■- price ^ ' . 

vfaat other schools : are doing' 

’ ' what the school has alwys; done • 

the test measures an area we. want measured 



19 » T^ scores from these fifth grade tests were' actually used^^.i^^ 
o (check; . iC;. j i,--.- , 









staff :-;curficullnh-plahhihg:;7sessl6ns' '>;k 

student pouhsellhg':-' and ' ' diagn'dsis'.-;y< ■ i’; 

pupil gr ou^ 



■vk'ii 






teacher ;:;eyaiuatlon'- ■ ; •. .C '-'-'' 

schdola‘'.v;''^>'r^ ' '• 






among schools 

pupil jppadih 
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sixth grade 



20. ; Please list by name the standardized tests administered last year to 
your sixth graders. For each test^ indicate the area of the 
instructional program being measured. 



Title of test 



Instructiohal 
area being; measured 





: 7 ^ ' .';;7 


. . ■ ■ \ . '7- 


. . ■ , ■ * jl" ■ 




. . ii- • 




■ ■'!, ’ . ■ ■ 
■■■■■;•■' 7^- ='■ 'il. ■ 




' ■ ’■ . i‘ 

. • • 7 • ■ , ’ ■' ■ xl'’. ■ . > 




7-;.' ■; aI- 

7 • -7 • ■ • • : j; ■ . - 



21. How were the majority of the above-named sixth-grade tests selected? 



state mandated 
district mandated ■ ■ 

'price ' 'r -V . ..'v';';' 

what other schools are doing -^7- 

what the school has always done 

the test measures an area we; want laeasured ■; 






ERIC 



22; The scores from these sixth; grade tests were act^^ 

(check;; all ‘''that;' a'p^ " ir ■'■■'■Y'"" 

staff curriculim-plarining' sessions f: 

,■ . student.' cpunaeilhg -;ahd,: di^^^ v; .r - \ 

V'. ;; 'imipii^ ]p-- ' 7 O’’-"-' 

•> '- :;bea;cher7'evalna^ V’- • ,• • . 7/ ' ■. ’'-'.V '■ 

\ ^ ' '.V. 

' ; '" ^'pupii'-CTadlng'^ \ ■- . ’■ ' /r-rv7.-: 



y'v- . 



^ •» 






X 



23. What do you think Is the attitude of the following people toward the 

standardized testing program in your school? (check one in each column) 



Attitude 


teachers 


guidance I 
personnel I 




principals 


provides necessary 
information 










it is required that 
the tests be given 










good measure of pupil 
progress 


• T ' ' 








good measure of curriculum 
success 










a good way of judging 
teacher competence 




. ■ ■ - 






forces teacher to 
teach to test 










a waste of time; 




■ 







24. Please elaborate on your own attitude toward the use of standardized 
tests in your school, 



O 

ERICV 



25 . You have just, 'learned thalt the average score on a standardized math 
■: achievement tes t given fp , air f third graders lah t May wa® 

: extremely low; (In ccnnparisoh: )to^ How yould you 

' explain this phenomenon? (check one) v'/ ' jir ' .. ‘ v' T' ' .--if' 



26-, 

!■’ ■ 

'■'I' 






_ really don’t know how to explain it' :■ | 

_ test , didnH measure wha,t was 'taught 
indicates a - shi'fT iih my- 

socidechnomic'-''staiusf:''';;v/''’:j;r.::'';:r-- 

^ 'would have t o examine the Ij t es t ; ;wi th my third gr ade 
'teachers/hefore I;jc 
4 '.other (plea 



;3' 

ii"' 















' Checks pneVof-'the^!^ fbllowihg'y.std^^eiitSv':^ 

.attitude ■ toward ^■nationai,^■norms,^'on■^'s tandardized^^ t^'ts 

^:^nat ioriai ; ; hbm ar e^ always e 1 1 ^fvt^ at^e^^r 

*]ridrms'>-r6r" making !, (decisions ..-reiar 



. .^;.i-^ :,what;-;are :','nat ipnal;,;nq:^ 

. . ■ : - ' ■ j U 1!:' n • ! « ’-U « 1 • V« '-i \ A 4 ^' ‘ i-. . Vi' Q n ' o *V ' n y* 1: < 



^ thq advantage of national ' norms '' is,.;it& 
aoiely:: 'oh.o'chbols.'.'-like' 



aoiely ;6h^h 
' ' lUt iqnai- noms , : eui*e^ mor e; . r e lia'ble , 






.- .y ■ 


•11 














.. . ■ .. 













}[iri •. ; 5 . 
'./I 
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APPENDIX C 



School 

District 

ELEMENTARY SCHOOL EVALUATION KIT 
Field Implementation QUESTIO?WAIRE #2 
To be aompleted by school -principal 



This questionnaire must be completed and returned to CSE before 
Booklet III: Selecting Tests can be sent. 

The questions below are being asked in order to enable CSE to evaluate the 
present edition of the Elementary School Evaluation KIT. No information 
will be released by CSE in any form that would allow identification of any 
particular school. 



DIRECTIONS; Read Booklet II and examine the contents of the box of 
goal card sets. Study the procedures for the Co/llective Viewpoints 
Approach (Method 2, ‘page 11) and administer the procedures to yourself 
twice with a two week period in between the administrations. Record 
your ratings in the table at the end of this questionnaire and then 
answer questions 1-7 below. ' 



1. List any student performance goals that you would like to see added to 
the list. 



2. List any goals other than student performance goals that you would like 
to see added to the list (i.e., goals pertaining to staff performance, 
school operation, community involvement, etc.). Please list specific 
goals. 



3. Were there any goals (as stated on the cards or in Appendix A) that 

seemed to be too similar to another? . 

Which ones? ' 



4. Were the goals organized in a manner appropriate for your school? 
How would you improve them? 



5. Would you have preferred the goals to be: (check one) 

more specific and more numerous? 

more general and fewer in number? 

6. Do you feel that the . goals and the organization of the goals are appro- 
priate for a wide variety of schools and. types of : students? 

yes no 




DIRECTIONS: Please read the remaining questions in this questionnaire 

and then administer the Collective Viewpoints Approach (Method 2, page 
11) to at least 10 teachers and 10 parents. Record each group’s average 
rating in the table at the end of this questionnaire and then answer 
questions 8-20 in this questionnaire. - Save the pink Tally Sheets and 
please return them with the questionnaire. 



7. List any student performance goals that your raters added by using the 
yellow blank cards. Also indicate the rating (1-5) given each "write- 
in" goal. 

Goal Rating ! 



8. List any goals other than student performance goals that your raters 
added (i.e., goals pertaining to staff performance, school operation, 
community involvement, etc.). Also indicate the rating (1—5) given each 
"write-in" goal. 

Goal Rating 



I 9. Was the system of rating goals contained in Method 2 easy for you? ^ 

I For your teachers? For your parents? _. List any difficulties 

I that were encountered? 

i . .... ■ ■ 

f ^ 

1 • • ■ ■■ 



I 

I 

i- 

V 



10. Would you, the teachers, or the parents have preferred some other way of 

rating the relative importance of goals? ■■ - . ■ . • 

If so, briefly describe it. 



11. Do you think other goals would have been chosen as "most important" if you, 

I the teachers, or the parents had used a method other than the one recommended? 

''Why? ■ • ■■ ■■ ^ ; '• ; ' ''• — 



12. In using the "Collective Viewpoints" method, did you learn anything new 
about the views of Teachers? - What? " 




Parents? What? 



1 

i 

! 

13. Did it confirm feelings you had about the viewpoints of Teachers? 
What? • 



Parents? What? 



14. Were the instructions understandable for the collective viewpoints 

method? Could you follow the procedure given? 

Could you do the computations easily? Were the results 

commensurate with the effort expended? - If not, why not? 



15. Did you try to combine the various viewpoints to get one set of goals? 

’ What method did you use? 



16. Did you do the random sampling of parents? ^ What problems in 

random sampling occurred in your context? ^ 



Is it easy? - ' ' ' 

Is it worth the effort? ' 

17. Did you do the stratified sampling of parents? 

What problems in stratified sampling occurred in your context? 



' Is it easy? ' ■ ‘ ‘ ^ ~ ~ 

Is it worth the effort? 

18. Briefly describe the reaction of the teachers and parents to being involved, 
(e.g., enthusiastic, threatened, frustrated, etc.). 



. , , . :101 



19. What is your overall reaction to Booklet II? 



20. Would you recommend procedures contained in Booklet II to other 
principals? If not, why? 



21. Other comments on Booklet II. 



Please fill out the table below. 

No. of teachers involved 

No . of parents involved ^ • 

Note: Please return all tally sheets (pink) with this questionnaire (don't 

worry about them being messy) • 



Educational Goals 


You: 

Rat 

1st 

rating 


r Own 
ings 

2nd 

rating 


Average 


Teachers ' 

Average 

Rating 


Parents * 

Average 

Rating 




Shyness-Boldness 












3. 


Neuroticism-Adjustment 


f 










^ • 


General Activity-Lethargy 














Dependence-Independence 












3. 


Hostility-Friendliness 






- 






• 


Socialization-Rebellioushess 














School Orientation 












3. 


Self-Esteem 














Need Achievement 












B. 


Interest Areas 














Appreciation of Arts and Crafts 












B. 


Involvement in Arts and Crafts 














Representational Skill in Arts and Crafts 












B. 


Expressive Skill in Arts and Crafts 












k. 


Arts and Crafts' Comprehension 








V 




B. 


Developmental Understanding of Arts and Crafts 


■ • - • 











A. 


Classif icatory Reasoning . 






.... 






B. 


Relational- Impiicational Reasoning 






• 


- •. ■ • • 




C. 


Systematic Reasoning 










V ■ 


D. 


Spatial Reasoning 






. . . 




' 


A. 


Creative Flexibility 


— 










B. 


• ; t ^ . 

Creative Fluency : 








^ ■ ... I 


... , 




\ 



Educational Goals 



1st 

r a f-ing 



lOA. Span and Serial Memory 



Your Own 
Ratings 



rating Average 



2nd 



Teachers ' 
Average 



Rating 



Parents * 

Average 

Rating 



lOB. Meaningful Memory 












IOC. Spatial Memory 












llA. Reading Comprehension of a Foreign Language 












IIB. Oral Comprehension of a Foreign Language 




• 








lie. Speaking Fluency in a Foreign Language 












IID. Writing Fluency in a Foreign Language 












12A. Cultural Insight through a Foreign Language 










■ ■ ! 


12B. Interest in and Application of a Foreign Language 










■ ‘.f 


13A. Spelling 













13B. Punctuation 










1 


13C. Capitalization 








. ' * i 




13D. Grammar and Usage 










i 


13E. Penmanship 










j 


13F. Written Expression 












13G. Independent Application of Writing Skills 








.... 




14A. Use of Data Sources as Pveference Skill 


• ■ 


, •• •, ) 








14B. Summarizing Information for Reference, 




’ ; 








15A. Comprehension of Numbers and Sets in Mathematics 










* * 


15B. Comprehension of Positional Notation in Math# 










! 

i 


15C. Comprehension of Equations and; Inequalities 


: 








j 


j 

15D. Comprehension of Number Principles 












16A. Operations with Integers 










' ■ ■ . i 


16B. Operations with Fractions 










• ' • ■ • ' ! 

i 


16C. Operations with Decimals and Percents 






r; 


1 •: . .. . 


• -1 


17A. Mathematical Problem. Solving 













17B. Independent Application of Mathematical Skills 
O 















- 1 



Educational Goals 


Your 

Ratin 

1st 

rating 


Own 

gs 

2nd 

rating 


4 

Average 


Teachers ’ 

Average 

Rating 


Parents' 
Average | 
Rating 1 


18 A. 


Geometric Facility 










1 

i 


18B. 


Geometric Vocabulary 










i 


19 A. 


Measurement Reading and Making 












19B. 


Statistics 

#> _ .. . . 












< 

o 




Music Appreciation 












20B. 


t 

Music Interest and Enjoyment 












21A. 


Singing 










' 1 

i 


21B. 


Musical Instrument Playing 










1 

1 

1 


2 1C. 


. \ 

Dance (Rhythmic Response) 










i 

1 


Ua. 


Aural Identification of Music 












22B. 


Music Knowledge 












23A. 


Practicing Health and Safety Principles 












23B. 


Understanding Health and Safety Principles 












i23C. 

t 


Sex Education. 












V 

i 

24A. 


Muscle Control (Physical Education) 












24B. 


Physical Development and Well-Being (Phys . Ed.) 












25A. 


Group Activity - SportsmanBhip 












i .. 

25B. 


Interest & Indepndt. Partcptn. in Sports and Games 












26A. 


Undrstndg. Rules & Strategies of Sports & Games 












26B. 


Knowledge of Phys-Ed. Apparatus and Equipment 












27A. 


Listening Reaction and Response to Reading 












27B. 


Speaking 












28A. 


Phonetic Recognition 












:28B. 


Structural Recognition 












P29A. 


Oral Reading 












29B. 

! 


Silent Reading Efficiency 










. 
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30A. Recognition of Word Meanings 



30B. Understanding Ideational Complexes 



30C. Remembering Information Read 



31A. Inference Making from Reading Selections 



31B. Recognition of Literary Devices 
31C. Critical Reading 



32A. Attitude toward Reading 



32B. Attitude and Behavior Modification from Reading 



32C. Familiarity with Standard Children's Literature 



33. Religious Knowledge 



Educational Goals 



Your Own 
Ratings 



1st 
rating 



2nd 

rating 



35D. Hypothesis Formation in Science 



35E. Operational Definitions in Science 



35F. Experimentation in Science 




Average 




Teachers ' 

Average 

Rating 



Parents ' 

Average 

Rating 



APPENDIX D 



School — 

City and State 

ELEMENTARY SCHOOL EVALUATION KIT 
Field Implementation QUESTIONNAIRE //3 

To he oompleted by edhqol princri'pal 

This tjuestlonnalre must be completed and returned before 
Booklet IV: Collecting information can be sent. 



The questions below are being asked in order to learn mpre about the 
environment in which the Elementary School Evaluation KIT is being 
Implemented. No information will be released by the School Evaluation 
Projept in any form that would allow identification of any particular 
schoql. 



DIRECTIONS I: Read the first section of Booklet III (pages 3-’12) and 

examine briefly Appendixes A and B. Please answer questions 1-5 which 
(Sre designed to check on how well this section is written. 



In regard to the first section of Booklet III (pages 3-12), please circle 
the appropriate number in each item below; 



1. CLARITY 

Was the section clear 
and easy to understand? 

2. ORGAl^IZATION 

Was the section well 
organized? 

3. APPROPRIATENESS 

Did the section include 
appropriate consideration 
in choosing tests? 

4. LENGTH 

Was the length of this 
section about right or 
was it too long or too 
short? 



VERY 

GOOD GOOD AVERAGE 



VERY 

POOR POOR 



12 3 



4 5 



12 3 



4 5 




12 3* 4* 5* 



*For this item only, if 
3, 4, or 5 io circled, 
please check one <^f the 
following: 

too short 

too long 



5. USEFULNESS 

Can you use the information 

contained in the section? 12 3 4 



DIRECTIONS lit Study Appei>dlx A of Booklet III rather thoroughly, then 
answer qqestlot^s 6-13. 



6. List below the top ^ goals as rated by your sample of teache rs uspd 
in connection with procedures In Booklet II. For each goal, selept 
the best test (using Appendix A of Booklet III) to use In assessing 
student performance relative to that goa,L in grade 3. 






test 



List below the top ^ goals as rated by your sample of paren t, used in 
conneotion with procedures In Booklet II. For each gpal, select the 
best test to use In assessing student performance relative to that 

goal in grade 5. 



I^al 



test 



8. Let's pretend that you are visiting with a school official ® 

nearby school district. During the conversation, he mentions that 
thinks a certain test Is an excellent test and that 

using it in your school. What questions would you ask him at that point 
(other than asking him to give you the exact name of the test) , In order 
to determine if you would use the test In your school? 



0 ^ 
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9. Using Appendix A of Booklet XII, select what you think is the best test 
available for assessing student performance for each of the nine goals 
and grade levels listed below. 

goal and grade level name of test 

Recognition of Word Meanings, grade 6 . 

Inference Making from Reading 

Selections, grade 6 — 

Attitude Toward Reading, grade 3 ^ 

■m 

Reading Comprehension, grade 5 ^ 

Reading Comprehension, grade 3 

Recognition of Word Meanings, grade 1 

Mathematical Applications, grade 6 

Arithmetic Operations, grade 1 

Operations with Integers, grade 3 



In regard to your use of Appendix A, please circle the appropriate nuinber in 
each item below: 



VERY VERY 

GOOD GOOD AVERAGE POOR POOR 

10. CLARITY 

Could you find the informa- 
tion you wanted easily? • 1 2 3 4 5 

11. USABILITY 

After you found the infor- 
mation, was it in a form 
that you could use? 

12. ORGANIZATION 

How well was Appendix A 
organized? 

13. TIME 

Could you locate the 
information quickly? 




O 
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DIRECTIONS III: The following questions pertain to Booklet III taken as 

a whole. 



14. Overall, would you consider Booklet III: (please check) 

Yes No 



Useful? 

Fills a need in your school? 

Complicated? 

Too long? 

Overly sophisticated? 

What is needed after using Booklet II? 

Too much effort for too little return? 

15. If you found that you were now using a test that was not suited to 

your school's particular assessment needs, would you be able to change 
to a more appropriate test? 

Yes No If not, why? 



16. Do you think Booklet III would help you in presenting a case for a new 
test to be used in your school district? Yes No. If not, why? 



17. If you are most Interested in a test that kids can xinderstand, you would 
choose tests on the basis of 

^asurement Validity 

^amlnee Appropriateness 

Administrative Usability 

Normed Technical Excellence 
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18. If you are most Interested in a test which will differentiate high ability 
pupils from low ability pupils you would choose on the basis of 

Measurement Validity 

Examinee Appropriateness 

Administrative Usability 

Normed Technical Excellence 



19. If you are most interested in a test which teachers can give and score 
easily y you would choose tests on the basis of 

Measurement Validity 

Examinee Appropriateness 

Administrative Usability 

Normed Technical Excellence 

20. If you are most interested in a test which really measures what it claims 
to measure y you would choose tests on the basis of 

Measurement Validity 

Examinee Appropriateness 

Administrative Usability 

Normed Technical Excellence 



21. What is your overall reaction to Booklet III? 



I 



22. Would you recommend procedures contained in Booklet III to other 
principals? Why? 



! 

i 

[ 



r 

I 



I 
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APPEaJDIX E 









;..i .i.. ■ .u. .1 






School 

.... .I . 



City and State 



ELEMENTARY SCHOOL EVALUATION' KIT 
Field Implementation QUESTIONNAIRE //4 



; • ./I J ' i.. ‘.i-. 






■ : 'V !i boii^Zeted hy echoot p2*in&Cpdt ' 



.Tj. 






, j t. ). : 



rt-- ri . n 






This 'questlbnhaire must be completed and returned ^before" 
Booklet V; Selecting Critical Need. Areas can be sent. 






/ :l ; . .i ! ' ' I * 



These "questions ‘are being’ asked in order' to’ - learn- more, about the 
environment in which the Elementary School Evaluation. KIT is 
being Implemented. No Infprmation will be released by the School 
Evaluation Project in any £orm that would allow •identification > of 
any particular school. 






DIRECTIONS I:’ Read the main text- of Booklet IV (pages .1-24) and answer 
questions 1-19 which relate to pages 1-24’ of the booklet. 




In regard to the section op PROCURING INSTRUMENTS (pages 2-7); 



1, Please indicate how valuable each part of this section was to you: 

Very Somewhat Not at all Eliminate 



Ordering Test — — 

Type of test — 

Scoring -service - __ — — 

Machine scorable answer sheets __ 

2. Were any areas of concern to you left out? YES NO 

If yes, please specify .. — 



In regard to the section on GENERAL GuiPELINES FOR TEST ADMINISTRATION (pages 8-10) 

3. Have you had problems in your testing program that would have been prevented 

if you had this information? YES NO 

4 . How useful will this section be in helping you with yo^r testing programs? 

very useful 

useful; should be left in booklet 

. •, ( ; ^ , not very useful, but probably should be left in 

: V no value; probably should be eli^nated ^ 

5. Were any areas of concern to: you left out? , YES_;_ . NO , 

If yes, please specify ; — ____ — : 



to the section on TEST SAMPLING PROCEDURES (pages 10—14) : 

6. How would you characterize your testing program? 

- always student-system assessment . 

r V .more s tudentrsys tern assessment ^than sampled-system 

.. V,, assessment. . -r.-i. 

more Si ^pled-systein assessment than student-system'. 

assessment 

always sampled-system assessment 







7. Do you think that you will implement a sampled-system assessment program 
in the future? 

' yes, by all means 

yes , , if pofifsible ; , . 

no , by ho means , i . 

8. What advantage does a sampled-system assessment have over a student- 

system assessment? - ^ — — 



9. If you were' interested in using a sampled-system assessment approach, do 
you think this section provides enough information for you to use this 
method in your school? NO ' : ' If; ho , ' what ^ind ' of ^ 

tion would you like to see added? ! ^ 1 1_ 



10. Is it clear to, you that we think, ^^8^ assessment is preferred 

to a student-system assessmeht for schbor evaluation? YES K J • 

Do you agree?' l YES, ,NO - • If ho, why?. 



In regard to the section on US ING NORMS (pages 14— 16)t; 

11. Have you ever attempted to perform a school . or program evaluation? 
YES NO . 



12. .Have you ever performed a school or program, pvaluat ion? YE^ 

If yes, did you use test norms as par t' 'of the evhlxiatioh? ' YES __ 
. What kind of test horms? ■ — — — — r::: — r r 



13. Hah the unavailability of school norms, ever prevented you from, per forming 
a school or program evalxiation? YES NO . . ■ >. i; - .. ' . . 



14. Have you - ever seen a table - of : school norms? .■ ,;YES 
■ for what test? ' [ ' ■ ' ’ - ' - ' • • 



^ ; :N0. v . - If .yes, 

■ . ; I i ; , ■ ■ : t :l 









15. Do you think test publishers should be encouraged to provide school norms? 

YES NO 

16. In regard to the subject of using norms, did this section: 

contain too much information 

contain about the right amount of information 
~~~ left out information on the following subjects: 



17. Check one of the following statements that best reflects your attitude 
toward national norms on standardized tests: 

what are national norms? 

national norms are always better than state 
or local norms for making decisions relative 
to the instructional program. 

the advantage of national norms Is that they are 
” based solely on schools like my own. 
national norms are always more reliable. 

other (please specify) — 



Atlas of Scores 



This section of Booklet IV is being revised and the changes will appear in 
the final version under the heading "Differentiated School Norms." The ideas 
expressed in the first two paragraphs of this section (page 17) remain the s^e; 
the changes take place in the way we are atteii 5 )ting to operationalize these ideas. 
We have discarded the notion of an "atlas of scores" which would be a supplement 
to regular norm tables. Thus, Appendix A will be dropped from the booklet and 
replaced by our substitute system. 

This system would be similar to the norm tables In most technical manuals j 
but the important difference is that there would be different norm tables for 
different "types" of schools, and these tables would contain school norms Instead 
of pupil norms. Thus, If we are successful In our endeavor, it would be possible^^ 
for you to find out how your school compares with other schools that are similar 
in terms of some as yet unknown characteristics. This comparison is likely to 
be stated in terms of a percentile rank, such as: the performance of your third 

graders in science fell at the 83rd percentile when compared to third graders in 
similar schools. Such a system of "differentiated school norms would be an im- 
portant development in improving school evaluation and school accountability. 
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18. Taking into account what is presented above as well as what is presented 
on pages 16 and 17 in Booklet IV, how important for school evaluation 
do you think the development of "differentiated school norms" is? 

very important 

somewhat important 

not important 

no comment 



Change Scores 



Our own review of this section has led us to conclude that it is inappro- 
priate for a needs assessment evaluation. We clearly had program evaluation 
in mind when we wrote it, and this material will appear in a more appropriate 
booklet. Therefore, there are no questions about change scores. 



Communicating Evaluation Results 



Again, our review of Booklet IV has shown that this section is slightly 
misplaced. We are keenly aware that this is a very important topic in this 
day of community involvement. However, the needs assessment evaluation has not 
yet been concluded in Booklet IV, so it is premature to talk about communicating 
evaluation results at this phase. Instead, we will place this section at the 
end of Booklet V, the last booklet in the Elementary School Evaluation KIT: 

Needs Assessment. Therefore, there are no questions about communicating evalxia- 
tlon results, at this time. 



The following question pertains to Booklet IV as a whole. 
19. What is your overall reaction to Booklet IV? 





DIRECTIONS II: 

The developers of this KIT, the School Evaliiation Project of the UCLA 
Center for the Study of Evaluation, are continuing the research needed for the 
development and refinement of future booklets related to the Elementary School 
Evaluation KIT series. Your assistance in this research process is being 
sought at this point. The attached questionnaire has been developed to provide 
valuable information to the School Evaluation Project regarding how elementary 
school principals view test results. Please follow the directions contained in 
the questionnaire and return the completed questionnaire with your Booklet IV 
Questionnaire. Thank you for your assistance. 
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APPENDIX F 



School 

City and State 



ELEMENTARY SCHOOL EVALUATION KIT 
Field Laplementatlon QUESTIONNAIRE #5 



To he completed hy eohool principal 



Theae questions are being asked in order to learn nore 
about the environaent in vhich the Eleaentary School 
Evaluation KIT is being iapleaented and hov well it is 
functioning. No information will be released by the 
School Evaluation Project in any fora that would allow 
identification of any particular school. 





In regard to the aectlon on Methods and Rules for Selecting Critical Need Areas 
(pages 2-8) : 



1. I understand what is meant by models and decision rules (check one). 
Yes No 



2. Did pages - make it clear what the outcome of implementing Booklet V is? 

most definitely 

I think so 

______ I am not sttre 

most definitely not 



3. Was it valuable to have an explanation of atkl examples of decision models 
and rules (pages - )? 

______ very useful 

useftLl , should be left in booklet 

______ not very useful y but probably should be left in booklet 

not useful, probably should be eliminated 



A. Would you consider the section on extraneous decision rules (check one); 

Yes No 



Very poignant - It indicates the need to 

have meaningful decision 
models 

Useful - It was a good reminder of 

the pitfalls into which a 
principEd might fall 
Irrelevant — It added nothing to my 

understanding of decision 
models and rules 
Insultingly foolish - 



5. How useful was this section to your understanding of the Booklet*s decision 
model and rule (check one) 7 

useful, leave it where it is 
useful, but should be put into an Appendix 
■ not very useful, but probably should be left in 
______ to value, probably should be eliminated 



o 
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In regard to the section on A Comprehensive Declalon Mod el and Rule (pages 8-15): 



6. Did the Booklet give an adequate explanation of the variables in the decision 
model (pages - )? 

Yes No 

If No, which varlable(s) was Cwere) not explained adequately? 



7, Do you think that there are some important variables which should be inclxided 
in the decision model but are not? 

Yes No 

If Yes, what are the variables that you think should be included. 



In regard, to the section on Implementing the Decision Itodel (pages 15-23) : 



8. Was it valuable to have the procedures for computing priority values repeated 
for you? 

Yes _____ No . 

9. Did you have difficulties In following the procedures for computing priority 
values? 

Yes No 

If Yes, for which of the following columns did you have trouble, and what 
was the nature of the difficulty? 

■ Coajumn 1 " • . ' • - ■ . • • ■- iJ— — — 

' Column 2 ■ ^ • • . . 

Column 3 .. ' - 

^ Column 4 ' • - . ■ ' ■ , ; : ^ ''' ' ' - ' - ■ ; - — 

• Column 5 ' ■ • ■ 

Column 6 • ' ' ' • . ■ ' ■ ' • ' - — 

__ Other difficulties (please specify) - 



10. Did you have any difficulties In using Appendix 1 to obtain a school per- 
centile score? 



Yes No 

If Yea, which of the following were involved (check all that apply)? 

Getting a roster of scores 

Obtaining the mean raw score 

Finding the appropriate table in the test manual to convert 

raw scores to pupil percentile scores 

Converting the mean raw score to a pupil percentile score 

Using Table U to convert the pupil percentile to a school 

percentile 

Other (please specify) 



11. Would it be of any value to you if it were explained how the table in 
Appendix 1 was derived? 

Yes No 



12. Would it be of any value to you if it were explained where the numbers in 
Tables 1, 2 , and 3 were from (check one)? 

Yes , it is necessary 

Yes, it would be interesting and helpful 

No , it would be confusing 

No, the numbers stand by themselves 



U. Did you actually compute priority values for any goal areas? 

Yes No 

If Yes, please fill in the following table, or send a copy of Form B 

Probable 

Goal. School School Bated Increase in Priority Rank 

Type Percentile Importance Utility Value 



In regard to the section on Implementing the Decision Rule (pages 23-25): 



14. Given the test :;esult8 and rated iaportance of certain goal areas (those 
goals where the decision was made earlier to gather student performance 
data), could you have determined the top priority goal areas without 
using the procedures discussed in this Booklet? 



yea No 



15. If you actually computed priority values for some goal areas, did you 
agree with the goal arcs that was selected by the decision rule as the 
most critical goal area? 

Yes No 

If No, why did you disagree? 



In regard to Booklet V taken as a whole: 



16. 



Overall, would you consider Booklet V; 

Useful 

Fills a need in your school 

Coo^licated 

Too long 

Overly Sophisticated 

Nhat is needed to complete a needs assessment 
Too much effort for too little return 



Yes, Ho 



17. Would you recommend the procedures in Booklet V to other principals? 

Yes No 



If No, why not? 



18* If you wish, you may add any comments that 
the questionnaires. ' ' ' ' - ' ’ ' ^ ~ ' 



were not solicited by items in 
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In regard to the complete ESS KIT; Needs Assessment ? 



19. Overall, would you conaldcr the KIT: 

Useful 

Fills a need, in your school 

Complicated 

Too long 

Overly Sophisticated 

Too much effort for too little return 



Yes Ho 



20. Would you recommend the KIT to other principals? 



Yes 



Ho 



If No, why? 



21. Would you ssy that the KIT provides net ter procedures for needs assessment 
evaluation than you had been using? 

Yes Ho 



22. How does the KIT compare to other procedures for needs assessment evalua' 
tion that you know about? 

much better 
______ better 

about the same 

poorer 

much poorer 

don’t know about any other procedure 

no comment 



23. Did you have anyone help you 






Booklet 

2 

3 

U 



Yes 





Ho 



Who 



The last set of questions are asked to determine the changes that may have 
occurred since the implementation of the KIT. Please respond as you now see 
the answers, not as they once were. 



24* Using the numbers 1, 2 , and 3, indicate on the following list the top 
three groups that place the most pressure upon you to institute educa- 
tional reforms and/or new programs (1 most pressure): 



Church or religious groups 
Business organisations 
Women's groups 
Other professional groups 
Veteran's organisations 
Property owners' association 
Teacher organisations 
Student groups 
District administrators 
Parent-teacher associations 



25. Please rank in order of importance the following considerations relative 
to deciding on the allocation of funds for next year's programs in your 
school (1 * most important): 

State mandates 

Local Board of Education mandates 

student needs as indicated by standardised tests 
student needs as observed by teachers 
suggestions from parents 



26. You are suddenly given $500.00 to spend on any one instructional area 
in your school. Please describe briefly how you would decide where to 
spend the money. 



27* How will Che majority of the. tests used In next year*s testing programs 
be selected (check one): 

______ state mandal'd 

______ dilstrict mai.tated 

_____ price 

_____ vhat other siiools are doing 

vhiit the sch > 1 has Eulvays done 

the test meaires vhat ve want measxxred 



28. If you were Interested In lecnlng of the attitudes of your teachers toward 
a particular Issue regarding tje Instructional programy how would you go 
about doing this? (check all hat apply) 

_____ contact teaches individually 
______ call meeting 3 teachers 

_______ contact teach e organizati<m(s) representative(s) 

______ occasion does 3 t arise where this is necessary 

______ other (please ;>ecify) 



29. If you were Interested in leanng of the attitudes of the community toward 
a particular Issue regarding >cr school's instructional program, how would 
you go about doing this? (chet all that apply) 

______ britig up at meting of FEA or similar parent organisation 

_______ check with teeters 

_______ contact a few | rents 

. occasion does it arise where this is necessazy 
_______ other (please escify) 





APPENDIX G 



SOCIOMETRIC DEVICE 



Name the three people in the school whose opinion on school problems 
you most respect. 

A. 



B. 



C. 



Name the three people in the school whom you think have the greatest 
influence on what goes on within the school. 

A. 



B. 



C. 
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APPENDIX H 



ELEMENTARY SCHOOL EVALUATION KIT: 
NEEDS ASSESSMENT 
BASIC CASE STUDY DATA 



IDENTIFYING DATA 

County 

School 

Principal 



Date 

District 

Address 

Superintendent 

Immediate Supervisor 



Chairman, Board of Trustees 

Type of District (circle): unified, elementary 



(Name) 



Grades Taught : 

District Enrollment as of September 30, 1970 



A. 



THE COMMUNITY 

1. Population and characteristics. 

a. Population of city or town in 1960 

b. Population of school district in 1960 

c. Populatipn of school attendance area 1960 



1970 

_ 1970 _ 
1970 



B. SCHOOL ATTENDANCE AREA 

1. Major occupations of parents in school attendance area: 

2. Community organizations, 
a. Churches. 

1) Influence: strong moderate negligible _ 

2) Youth programs: many relatively few non 
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b. Service clubs. 

1) Influence: strong moderate negligible 

2) Youth programs: many relatively few none 

c. Public recreation accessible to school population, 
(indicate numbers) 

1) Playgrounds 

2) Swimming pools 

3) Organized athletic leagues 

4) Other (list): 



d. Youth Organizations. (check if available) 



1) 


Youth center 


2) 


YMCA 


3) 


YWCA 


4) 


Scouting Organizations : 




Boys 




Girls 


5) 


Other (list): 



3. Describe any community groupings or patterns within your school 
attendance area which in your opinion directly affect your school 
program. (Economic, social, racial, or religious groupings). 




1 
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C. BOARD OF TRUSTLEo AND DISTRICT ADMINISTRATION 

The following are Included in a series of standards that have been 
developed jointly by representatives of the National School Boards 
Association and the American Association of School Administrators, as 
published in the pamphlet, "The School Board Member in Action." All 
Boards of Trustees should be familiar with these standards and use them 
as a guide in the performance of their duties. Please check below the 
term which, in your opinion, most accurately describes the degree to 
which these principles are followed by all members of your Board of 
Trustees. 

1. The Board distinguishes between its responsibilities as a policy- 
making body and the superintendent’s responsibility as its 
executive officer. 

. yes generally no 

2. The Foard, through its function of local control, guarantees: 

a. Freedom for expression of local ideas. 

yes generally no 

b. Freedom for tailoring school programs to fit local needs. 

yes generally no 

3. Since the laws assign powers, duties and responsibilities to the School 
Board as a whole, the Board functions as a unit in all matters, rather 
than assigning responsibilities to individual members. 

yes generally no 

4. The Board does not limit itself to business and financial affairs, 
but also considers educational problems and policies brought to it 
by the superintendent . 

yes generally no 

5. The Board has established personnel policies wherein: 

a. The superintendent represents the Board in dealing with the 
staff. 

i yes generally no 

b. - The superintendent makes all personnel recommendations for the 

Board’s consideration, including employment, promotion and dismissal. 

' yes generally no 

[ 

); 

t' 

r 

X 

h- 



t 
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6. Board pollcito provide that prob ems or complaints from the community 
or staff members are routed through the proper administrative officers. 

yes generally ^ no 

D. THE CERTIFICATED STAFF IN YOUR SCHOOL: 

1. Certificated Personnel (reported in full-time equivalents) 

Men Women Total 



1. Administration 

2. Supervision 

3. Counseling 

A. Health Services 

5. Testing 

6. Student Activities 

7. Instructional Materials 

8. Other Non- teaching Assignments 

9. Home Teaching 

10. Regular Instruction 

11. Special Teachers for Physically 

Handicapped 

12. Special Teachers for Mentally 

■;'.etarded 

13. Other 



14. Totals 

2. Credentials by which staff is employed , including administrators. 

The "years of service" column refers to service within the district . 
(List by fractions where more than one credential is necessary.) 

Years Pro- Stand- Deslg- 

of vision- ard Special nated Super- Admlnls- 

Service al Creden. Sec. Services vision tration Totals 



1 



5 or more 



Totals 

3. How many of your teachers are Inexperienced (i.e., first year in 
teaching) ? 

Men 

i 

i 

Women 



Total 




131 
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4. Distribution of staff by age and sex, including administrators. 

Age 20- 26- 31- 36- 41- 46- 51- 56- 61- 66- 

Bracket 25 30 35 40 45 50 55 6 0 65 up Totals 



Men 

Women 

Totals 



5. Academic preparation by sex, including administrators. 

No A.B. + 40 Ph.D. 

Degree A.B . (no M.A.) M. A. Ed.D. 



Men 

Women 



1) What procedure is followed in making teaching assignments? 



6. Professional growth data over the last three-year period: 

No . of Teachers No. of Administrators 

Attending Teaching Attending Teaching 



College Classes 
College Sum. Ses. 
Workshops 
(2 or more days) 
Extension Courses 
Travel (if credit 
allowed) 

Work Experience (if 
credit allowed) 



t 7. Community participation: 



Please comment on the extent of teacher parent interaction. 
How many teachers live in the school attendance area? To 
what extent do parents and teachers participate together in 
community activities? 




1 ;<■'> 
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8. Teacher and adninistracor inembtrships in profes' ional organizations: 
Organization No. of Members 






E. THE STUDENT POPULATION 

1. Data gathered on students: 
a. IQ measured by 

Fori;' 



Name of Test 



Date Given 



b. 



Low Ql Median Q3 

Reading levels, measured by 

Name of Test 



High 



Form 








Date 


Given 


Low 


Ql 


Median 


Q3 




High 


Arithmetic 


levels , 


measured by 














Name of 


Test 




Form 








Date 


Given 


Low 


Ql 


Median 


Q3 




High 



2. Sociological characteristics of student body: 

a. Describe any racial or ethnic groups that are important enough 
that they must be recognized to understand the operation of 
the school. 




-7- 



b. Number of students who are transported in school buses 

This total represents Percent of the student body. 

3. Stability study for oast three years: 

School Year School Year School Year 



Class Entering 1967-1968 1968-1969 1969-1970 



a . 


Fall opening enrollment 
Transfer in 




b. 




c • 


Add for sub-total 




d. 


Transfer out 




e . 


Subtract for sub-total 




f . 


Spring closing enrollment 









4. Student-teacher ratio; 
a. Gross ratio: 

Total student body on October Report = STR (Gross) 



Total certificated staff 

b. Actual student-classroom teacher ratio: 

Total student body on October Report = STR (Actual) 



Teachers engaged in classroom instruction 
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F. SCHOOL FINANCE 

1. Assessed valuation of district 

2. Assessed valu tion per student in the district 

3. Elementary ADA 

4. Current tax rate 

5. Bond tax rate 

6. Bonded indebtedness 

7. Current expense by budget classifications: 

Total per ADA 

a. Administration ‘ 

b. 1) Certificated salaries of 

instruction — 

2) Other salaries of 

instruction 

3) Other expenses of 

instruction 

c. Auxiliary services 

d. Operation of school plant 

e. Maintenance of school plant 

f. Transportation of students 

8. Average current expenditure per 

student (last three years) ^ 

9. Materials of instruction - statistics. (Revise listing as necessary 
to fit local budgetary procedures.) 

Expenditure Average Expenditure 

Materials of Instruction per yr. for pas t 3 yrs. per ADA 

Text and Supplementary Books 

Periodicals, Library 

Library Books 

Audio-Visual Materials 



O 

ERIC 
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APPENDIX I 



INTERVIEW QUESTIONS - PRINCIPALS 



KIT 

1. Why are you participating in this program? 



O 

ERIC 



2. What do you plan to do with the KIT? 



3. a. What are your strategies for accomplishing this? 
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KIT (continued) 

b. Are there any strategies that you have considered but rejected? 



4. What do you expect will be the outcome? 



V 

i 

I School' Setting 

I ■ * 

la. Describe the Informal structure of your school. 

f 

f 



O 

ERIC 
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School Setting (continued) 



b. Who are the most influential teachers? 



c. Are there recognizable cliques? 



d. Do these cliques influence your school in any way? 
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School Setting (continued) 

2. How would you describe your school? 



3. Describe the educational program of your school. 



Principals 

1. What is your view of the KIT? 



O 
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Principals (continued) 

2. Have you ever participated in any such program before? 



3. How long have you been principal at this school? 



4. Assess your success as a principal in your school. 



I 

} 



/ 
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Principals (continued) 

5. What exposure have you had to the KIT? 



6. Please comment on the level of district support for your participation 
In this project. 



Parents and Community 

1. Please describe the parents who reside In your attendance area. 




Parents and Community (continued)" 

2. What is the community's view of the school? 
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3. What accounts for this view? 
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APPENDIX J 



CSE Evaluation KIT 
Teacher Interviews 

Name School 

1. How did you first become acquainted with the KIT? 



2. What do you understand to be the objectives of the KIT? 



3. Do you have any comments about the way the KIT was Introduced into 
the school? Were the Instructions clearly explained? Are there 
ways Its Introduction could have been Improved? 



4. Do you have any unanswered, questions about the KIT? 



5. Were the results of your school's participation In the evaluation 
project explained to you? 



6. What, if anything, does the school plan to do as a result of its 
participation In the CSE Evaluation Project? 



7. Did you learn anything from your participation in the evaluation 
project? 
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8. What opinions did the other teachers have about the evaluation 
project? 



9. Is there anything we have not discussed upon which you would like 
to comment? 



I 
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